There is a fundamental cultural reason for this, and the idea that The Past was more magical than the present goes back to the very earliest fantastical stories.
As Mark Olson points out, it is an unavoidable fact that the world we live in does not have any magic in it. For pre-modern peoples, there were things that they could not satisfactorily explain, like the sun's heat, the changing of the seasons, or the phenomenon of lightning. On the other hand, there were no anthropomorphic gods walking among them. Zeus and Odin and Shang-Ti never appeared to their worshipers, so if those people were to believe that such gods really did take human form and really did work miracles on Earth, they had to believe that there had been a time in the past in which gods and magic were much more active.
So the natural human viewpoint with regard to magic is that it was more powerful in the past. For a writer like Tolkien, who (at least originally) was intentionally creating an alternative mythic past for the real world, it was thus inevitable that the grandeur and magic of the cosmology had to be decreasing over time, because the endpoint was today's magic-less world. Of course, not all writers are specifically trying to create such a mythopoeia, but there is still a natural inclination to make the past more magical than the present, because it makes the fictional world more relatable. Having a fictional world where the magic is decaying, where wizards and fantastical beasts are almost extinct, makes that world seem more like the real world, and it makes the viewpoint of the characters who inhabit that world more accessible to a real-world readership.
Urban fantasy is, in some ways, a reaction against the idea that magic has been consistently in decay. This sub-genre posits that the magic is still here in the modern day. This opens up many new avenues of exploration and opportunities for interesting plots. However, for many readers, it is harder to maintain suspension of disbelief, because we know that there isn't really any magic in the today's world.
Quoting myself from here:
The idea that magic was much more plentiful in the past is very common, both in fantasy fiction and mythology. It makes sense that myths would tend to develop this way. While there are things in the everyday world that seem magical in varying degrees (earthquakes, lightning, the bodies of the heavens), the physical forms of gods are conspicuously absent. So there is posited (either implicitly or explicitly) to have been an age where magic was more common, and the gods were closer. There might or might not be a new age of magic coming in the future, perhaps at the end of the universe.
Fantasy writing, including my own, tends to follow this formula. One reason for this is that it’s familiar from myths and folklore; it’s been a convention of the fantasy genre before the genre even existed as such. Another reason is that fantasy is often set is a world that is supposed to approximate some epoch of Earth’s past—culturally or technologically. If too much magic was available, it would change the setting drastically. (This has been a real issue in some Dungeons & Dragons campaigns I’ve been involved in; they featured magical versions of the Industrial Revolution.) Yet a writer generally wants there to be enough powerful magic to tell a good story. A natural way of resolving this tension is if powerful magic exists, but it is hidden away; and even if it is uncovered, it cannot be duplicated.
Some authors have thought through the implications of the gradual decay of magic rather carefully. Others tend to accept it as a part of the standard fantasy setting without much explicit discussion. My father, when he was reading the Silmarillion, commented that he liked how, in Tolkien’s world, a great work of magic (like the Silmarils, or the Rings of Power) could only be created once; there would not be enough power left in the world to recreate such things. And it’s worth noting that the continued use of the Rings of Elves was a major driver of the elven economy in Middle Earth.
