Yes, they really were, although that all three were judges of the dead is a later development.
First, in Homer's Odyssey only Minos is the judge of the dead:
"“There then I saw Minos, the glorious son of Zeus, golden sceptre in hand, giving judgment to the dead from his seat, while they sat and stood about the king through the wide-gated house of Hades, and asked of him judgment.
Od. 11.568-571, trans. Murray
As you can see, Minos is concerned with judgment and justice, both important Greek concepts for the divine law of Zeus (the roots are themis and dikē respectively).
Pindar, about ~200+ years later, knows of Rhadymanthus instead as the judge of the dead:
those who have died on earth immediately pay the penalty—and for the crimes committed in this realm of Zeus there is a judge below the earth; with hateful compulsion he passes his sentence. But having the sun always in equal nights and equal days, the good receive a life free from toil, not scraping with the strength of their arms the earth, nor the water of the sea, for the sake of a poor sustenance. But in the presence of the honored gods, those who gladly kept their oaths enjoy a life without tears, while the others undergo a toil that is unbearable to look at. Those who have persevered three times, on either side, to keep their souls free from all wrongdoing, follow Zeus' road to the end, to the tower of Cronus, where ocean breezes blow around the island of the blessed, and flowers of gold are blazing, some from splendid trees on land, while water nurtures others. With these wreaths and garlands of flowers they entwine their hands according to the righteous counsels of Rhadamanthys, whom the great father, the husband of Rhea whose throne is above all others, keeps close beside him as his partner.
Pindar, Olympian Odes 2.58-77
As you can see, Rhadymanthus is responsible for separating the good from the bad. There's no sense at all that he's partial, but rather the opposite.
Plato seems to have developed, or at least is the first to mention all three brothers acting as judges. Here impartiality is strongest. Zeus, fed up with the rich yet wicked judged in their fancy clothing and sycophants fooling the judges into thinking they were actually just, appoints:
sons of my own to be judges; two from Asia, Minos and Rhadamanthus, and one from Europe, Aiakos. These, when their life is ended, shall give judgement in the meadow at the dividing of the road, whence are the two ways leading, one to the Isles of the Blest (Nesoi Makaron), and the other to Tartaros. And those who come from Asia shall Rhadamanthys try, and those from Europe, Aiakos; and to Minos I will give the privilege of the final decision, if the other two be in any doubt; that the judgement upon this journey of mankind may be supremely just . . .
When a man's soul is stripped bare of the body, all its natural gifts, and the experiences added to that soul as the result of his various pursuits, are manifest in it.
Plato, Gorgias 523a and 524b ff (trans. Lamb)
So yes, from the earliest citation down through Plato's fuller account, the three judges are impartial judges who ensure the just are rewarded and the unjust are punished.