On reading the old Journey into Mystery comics, where Thor first appeared, it is not evident that the Asgardians are aliens. As far as the audience can tell, they are simply the Norse gods of myth.
When in comic history were the Asgardians revealed to be aliens rather than gods?
In the 1999 miniseries Earth-X (now officially designated as Earth-9997), the Celestials had long-ago implanted an egg within the Earth that would eventually hatch into a new Celestial. In order to protect the egg, they modified humans to have the capacity to develop superpowers either through evolution (thus Marvel's Mutants) or through some kind of trigger, such as the assorted characters with some kind of empowering origin story, such as the Hulk, the goal being that the empowered humans would protect the Earth from external (and internal) threats, unknowingly protecting the egg that would, eventually, destroy the planet when it hatched.
However, these empowered humans could theoretically develop Celestial-level powers, so the Celestials implanted a psychological failsafe where the expression of an individual's power was self-crippled so an individual with power only expressed it in a specific way at a specific level because that's what they thought their power was, not what their power actually is. So, theoretically, Peter Parker could be omnipowerful: a psychic like Professor X, healing powers like Wolverine, Reed Richards level genius, as stronger as the Hulk, so on and so forth, all dialed up to over 9000 (sorry, OVER 9000!), but because the triggering of his power was due to the bite of the spider, he psychologically limited himself to spider-like abilities (with some exceptions like his spider-sense).
Loki eventually discovered that humans weren't the only ones this had happened to: the Celestials did the same thing everywhere they implanted their eggs. He discovered that eventually beings who had been so manipulated, if they survived the hatching of a Celestial on their world, eventually metamorphosed into no longer being mortal, but in doing so became a mental blank slate when it came to their personal image, which changed based on the people around them, and then became self-reinforcing. So, the Asgardians weren't gods, they were a group of these aliens who encountered humans who believed they were gods, and thus they took on those personas, adopting their powers and psychologies to suit the beliefs of the humans, and this became self-reinforcing. Loki wasn't originally an evil trickster god, he was an alien that humans decided was an evil trickster god, and thus he became one. And the Celestial psychological trap slammed shut again.
(Incidentally, he also discovered Odin had been a normal human who had realized what was happening and exploited it, making the aliens unwittingly give him some of their power. This is why he underwent the Odinsleep whereas other Norse gods did not; he needed to do it to replenish the power he'd taken from them that made him immortal.)
Loki's realization was that all of Earth's gods and devils and demons were similarly other alien refugees who had taken on personas and believed that's what they were. He went around trying to fruitlessly convince beings like Mephisto that he wasn't inherently a manipulative evil demon and certainly didn't have to stay that way, but the beings who had changed now had such a strong self-image, because of the Celestial failsafe, that he was unsuccessful trying to do so.
So, long answer, that was one example where the Asgardians (and the Olympians, and every other pantheon, god, devil and other "supernatural" creature) were revealed to really just be aliens who had, unintentionally, been forced into those roles.
