While the Rambam does not regularly list his sources, he bases himself on sources. The many commentaries on the Rambam do discuss his sources. A good approach could be to learn Mishneh Torah from a book with such commentaries, or on Sefaria, where you can click on a passage and then read what the commentaries have to say.
In this case, I will skip the commentaries and instead show Rambam aligned with Rif on Yevamot and with Yevamot 61b.

See how each of his phrases match an idea in the Mishnah and gemara, and then while a slight logical leap may be involved, it is not as great a leap as you might think. Rambam's continuation with שָׁהֲתָה עֶשֶׂר שָׁנִים וְלֹא יָלְדָה similarly matches Rif's highlight of points in the gemara.
Also, as mblock wrote in his answer, you left out the end of the quote, "unless he has already fulfilled the mitzvah of being fruitful and multiplying, or he has another wife with whom he can father children."
The Mishnah there writes:
מַתְנִי׳ לֹא יִבָּטֵל אָדָם מִפְּרִיָּה וּרְבִיָּה אֶלָּא אִם כֵּן יֵשׁ לוֹ בָּנִים. בֵּית שַׁמַּאי אוֹמְרִים: שְׁנֵי זְכָרִים, וּבֵית הִלֵּל אוֹמְרִים: זָכָר וּנְקֵבָה, שֶׁנֶּאֱמַר: ״זָכָר וּנְקֵבָה בְּרָאָם״.
MISHNA: A man may not neglect the mitzva to be fruitful and multiply unless he already has children. Beit Shammai say: One fulfills this mitzva with two males, and Beit Hillel say: A male and a female, as it is stated: “Male and female He created them” (Genesis 5:2).
See how the gemara takes it from there.
So, if a man doesn't have a children yet, and will only be practically marrying one wife, if she is old he is effectively neglecting such a mitzvah.
I understand from your posting an answer to your own question shortly thereafter that the question was likely placed so that you could establish your answer, about how other authorities disagree - though I am not sure that they do disagree. But yes, there are often several ways of interpreting underlying sources. Thus, there is a logical leap. But that doesn't mean that the Rambam stands with no basis in "Torah and Halachah".
Also, I'm not sure your answer even contradicts what the Rambam wrote. When you cited Rabbi Touger that "The Ramah (loc. cit.:5) and other authorities, however, differ and explain that as long as relations are carried out in an ordinary manner, having relations ... does not violate this prohibition.", the preceding sentence was "From the Shulchan Aruch (Even HaEzer 23:1), one can infer that sexual relations ... are considered as emitting wasted seed, one of the more severe prohibitions of the Torah." That is, the Rama is arguing with the inference made by Rav Yosef Karo in Shulchan Aruch (about spilling seed), not arguing with anything that the Rambam wrote in Hilchot Ishut (about that one shouldn't marry because he'll be thereby neglecting the precept of having children).