Technically, she cannot get divorced before she is old enough to actually understand what is going on and what "separation" means. Thus, a baby under the age of three could never become divorced as she can not accept a get and understand the entire matter just like a shoteh. See the Rambam on the halachos of marriage of a ketana and that a ketana leaves by miun (refusal) and not divorce. Thus, she could never have been divorced until she is an adult (above bas mitzvah).
I had remembered the Rambam incorrectly. Rambam Sefer Nashim Hilchos Geirishin (divorce) Perek 2 Halachos 18, 19
18:
If the father married off an underage daughter and she was divorced while still a ketana, the father accepts the get on her behalf and when the get "reaches his hand" she is divorced. If she is divorced while she is a na'arah, whether the get "reaches her hand or her father's hand" she is divorced. A na'arah who is betrothed cannot become a shaliach to accept her get from her husband while her father is still alive. However, the father can become a shaliach to accept the get for his daughter who is betrothed whether she is a ketana or a na'arah.
19:
If her father married her off while she was a ketanah and then died. If she can tell the difference between the get and something else she is divorced as soon as it comes into her hands and if not, she is not divorced until she is capable of distinguishing, and if she was divorced (received a get while too young) she is not divorced.
Thus if the father is available to receive the get while the daughter is under three, she is divorced and asur to marry a cohen even if she is still a besulah. If the father died while she is still too young to accept a get and know what is happening, then she must wait until she is old enough to understand the entire concept of divorce and what the get is.
The Rambam speaks of the halacha of miun in Hilchos Ishus Perek 4 Halachos 7, and 8. If a ketanah is an orphan or the father no longer has the authority to marry her off (because he has already done so and she was divorced or widowed) then she must be old enough to have the daas to accept the marriage. Betwee the ages of 6 - 10, we test her to see if she is mature enough to understand what is happening. If she is, then she is married but can "refuse" when she becomes a na'arah and cancel the marriage (as he explains in Hilchos Geirishin). She must state in front of two witnesses that she refuses to stay with the husband and she leaves without requiring a get. The reason for this is that the marriage was not a "complete marriage" from the Torah but only through "divrei sofrim" (rabbinical decree) which is only valid if she stays with the husband after she becomes a gedolah and they "finalize" the marriage completely. They do not require getting "married" once she is adult but if she does not want to stay married to him she must perform the refusal and leave without a get.
Similarly, if the mother and brother marry her off after the father dies (and she is a ketana) she can leave with miun.
Also if she ended the marriage by miun (refusal) then she is not a gerushah and it is as if she had never married. Miun occurs when she becomes a na'arah (12 - 12.5). Note that she would probably be asur to a kohen gadol as not a besulah even though she had "never" been married.