Of course, this situation wouldn't remain for long. Once the first edition of A&D debuted, the Player's Handbook included 30 spells, ranging from write (transcribe a spell that is too difficult for you to understand yet, so you can at least have a copy in your spellbook for later) to magic missile (the venerable low-level damage-dealing spell) to spider climb (climb up walls and ceilings like an arachnid). As the years and editions rolled on, this situation continued to expand. Unearthed Arcana added new spells to the list, then 2nd edition updated the list and added new spells with its Tome of Magic supplement. 3rd edition similarly added spells in nearly every hardcover character-centric book -- Complete Arcane, Stormwrack, Dungeonscape, you name it, there were new spells.
This presents a bit of a problem. On the one hand, players love the variety. Having lots of spells provides you with the opportunity to flip through a hundred pages of wonders and find the ones that really speak to your character. If you want to play a wizard who's a master of cold magics, you pick and choose all of the cold spells from the many lists. On the other hand, though, you wind up with choice paralysis and non-choices. Some spells are clearly advantageous -- even looking at tiny cantrips in 3rd edition, acid splash is often slightly better than ray of frost, simply because acid splash is a conjuration spell that bypasses spell resistance (you're making a ball of actual acid, instead of shooting a ray of magical cold -- a distinction that really winds up being hair-splitting in a lot of cases). And there are so many spells that unintended synergy is a real problem, and you can almost always find a magical spell that does exactly what you want.
With so many spells, there's little incentive to "think outside the box," to find creative uses of existing spells or look for nonmagical solutions to problems. Instead, magicians are encouraged to simply find just the right spell for any problem. Plus, if there's a spell for everything . . . what does your magician create? There's less incentive to make your own spells -- and leave your mark on the world! -- when all the spells you could ever want already exist.
Naturally, this sells lots of books. Having spell compendiums is a good way to generate revenue because players are primed to accept that anything published is always, de facto, included in the game. If it wasn't "balanced and appropriate," it would never be published, right? D&D went down this road pretty hard late in 3.5 edition, with Eberron explicitly trying to include everything published in every sourcebook. It sells, so it's in the publisher's interest to do it.
What happens when you go the other direction? Pare down your spell list. Remove non-essentials. Give spells a specific niche and make them fit a theme. This lets you tune your magic to support your campaign setting. If you remove detect evil, you can create a campaign setting with more moral ambiguity. Remove all evocations and you can have a game that doesn't use flashy fireballs and lightning bolts, so wizards will focus on other ways to get the job done. Obviously in Dark Sun, you get rid of create water. The very divide between cleric and magic-user spells from early D&D speaks to the idea that spells can function according to broad thematic categories. A great example of tightly theming spells is in Sean K. Reynold's Kickstarter project Goody White's Book of Folk Magic, which provides a selection of spells that work like curses and remedies from folk tales about witchcraft. This kind of book isn't a grab bag of twenty new ways to blow people up; it's a way to infuse your campaign setting with a specific flavor -- in this case, the home crafts, charms, and hexes that one would associate with witches both petty and powerful.
If you want a campaign in which spells are a bazaar and there's something for every occasion, you can throw a thousand spells into the mix and have something like the Forgotten Realms, where wizards have figured out ways to do everything worth doing (and a lot of things that aren't). If you sharply limit your selection of spells, you can build a campaign setting where your magic supports a common theme, or one where magic is a great problem-solver but only some of the time; your players have to use their wits to get out of scrapes, and they'll spend less time flipping through sourcebooks and more time thinking about how they could make their own named spells as a legacy!
If you're following in the footsteps of Vance's Dying Earth, in which there are just over a hundred known spells in all the world, less really can be more.