Knowledge of the One Ring was certainly not obscure, at least among the Wise (a very vague term). Gandalf states that the famous "One Ring to rule them all, One Ring to find them" verse was "long known in Elven-lore" (Lord of the Rings, Book I, Chapter 2, "The Shadow of the Past"). Gandalf further says that Celebrimbor, who made the three Elven-rings without input from Sauron,
was aware of him, and from afar he heard him speak these words, and so his evil purposes were revealed.
It seems certain, then, that Elrond, Galadriel, Círdan, and obviously Gandalf (the four custodians of the three Elven-rings who were still alive in the Third Age) were aware of the existence of the One. Presumably, then, others—including the descendants of Elendil, and the Stewards of Gondor, and those dwarves who were aware of the Dwarven-rings—were also aware of the existence of the One. But it seems that not everyone, even of those who knew about the ring, was aware that it was still around. When Elrond speaks of Isildur cutting the One Ring off Sauron's finger, Boromir exclaims:
I have heard of the Great Ring of him that we do not name; but we believed that it perished from the world in the ruin of his first realm. Isildur took it! That is tidings indeed.
(Lord of the Rings, Book II, Chapter 2, "The Council of Elrond")
It is quite possible, though I don't believe it is stated anywhere, that this is the typical position among those who know of the Ring's existence.
Among those who did know that the Ring was not in fact destroyed, it does seem to have been common knowledge that it was lost in the disaster of the Gladden Fields; but nothing had been heard of it for a couple of thousand years, and they didn't seem to think that it was important—until Gandalf discovered in 2850 that Sauron was still in the world.
By that time, however, Saruman had already begun to be corrupted, and it was in his interest to convince everyone else that the Ring, though perhaps not destroyed, was at least inaccessible—as Gandalf reports:
'"At the worst," said he, "our Enemy knows that we have it not and that it still is lost. But what was lost may yet be found, he thinks. Fear not! His hope will cheat him. Have I not earnestly studied this matter? Into Anduin the Great it fell; and long ago, while Sauron slept, it was rolled down the River to the Sea. There let it lie until the End."'
(Lord of the Rings, Book II, Chapter 2, "The Council of Elrond")
Saruman was the leader of the Council of the Wise, and considered extremely trustworthy by pretty much everyone. That's why Gandalf has such a problem reconciling his ideas about Bilbo's ring, based on his interactions with Bilbo, with Saruman's assertions.
Then, too, Gandalf (who is the only one aware of Bilbo's ring) apparently has other things to do besides think about the Ring all the time. He does set a watch to see if Gollum emerges from the mountains to try and get the ring back, but doesn't follow up on it:
But at the western edge of Mirkwood the trail turned away. It wandered off southwards and passed out of the Wood-elves' ken, and was lost. And then I made a great mistake. Yes, Frodo, and not the first; though I fear it may prove the worst. I let the matter be. I let him go; for I had much else to think of at that time, and I still trusted the lore of Saruman.
(Lord of the Rings, Book I, Chapter 2, "The Shadow of the Past")
It took until Bilbo left for Rivendell in 3001 for Gandalf's suspicions about his ring to harden into near-certainty that it was the One Ring. Then, of course, he had two tasks: to find evidence to back up his certainty, and to figure out how and why Saruman's claim came to be wrong. Gandalf explains in "The Shadow of the Past", and clarifies in "The Council of Elrond", how he managed both of these.