I figured it had to do with the fact that it was much more likely that Petunia would take in Harry if she felt like she was all he had. If Dumbledore had stuck around, I very much doubt he could have convinced her to take him over her saying simply "you take him, I don't want him". Also, it's possible that she and Vernon would have just freaked out if they saw Dumbledore. He clearly didn't have time for that. But I don't doubt for a moment that a watchful eye was always on Harry to make sure he did indeed get in safely.
To be honest, as awful as she is to Harry, I don't think she ever would have abandoned him. As Dumbledore said, she did take him in after all and didn't immediately take him to an orphanage. Actually, thinking of it, I would imagine that was the most selfish thing she would have ever done. You can bet your ass Vernon didn't want anything to do with him. And why would he, he hates everything he represents (actually I'm curious if he would know anything about magic up to this point. I can't fathom a reason Petunia would have ever told him until he demanded answers of where this child came from).
Anyway, we all know how timid Petunia is to Vernon and how his word is law. I would guess it took some serious guts on Petunia's part and determination to stand up to Vernon to tell him they were keeping Harry, which for anyone would be no small feat. Granted, his living situation was awful, but you wouldn't do that for someone if you didn't truly care (now whether it's for Harry or, more likely imo, Lily, I'm not sure).
Albus put his faith in his guess for how much Petunia actually cared about her sister and that she generally, deep down, is a good person.
Edit: not sure if my answer got a little bit too of topic. In short, I think Albus had a pretty good guess at how Petunia would react. His guess was then obviously verified by the events that happened. Albus is, after all, fairly wise ;)