My impression after being accepted and discussing this with senior faculty was the following:
A large, top tier graduate program receives applications. Some fraction of these (say a quarter) are entirely unqualified and will be discarded.
Another, very small group are fantastically qualified, with great research experience and superb academic credentials from the very best schools. These few will almost definitely get in, but may not accept.
Finally, the largest group consists of people with credentials that are no better or worse than yours (maybe they attended a better school, but their record is slightly worse, or their recommenders are more prestigious, but the letters are less personal). If you are in this group, you should do everything you possibly can to improve your application, but ultimately you're playing the odds.
The biggest obstacle you will face at the top tier schools is that they receive so many more applications that the acceptance rate might only be a few percent. This means the odds are very unfavorable that any particular top tier school will accept you as a solid, but not extraordinary applicant.
This is how it was explained to me anyway. So, I don't think your school is a serious handicap, but it probably does exclude you from the "Oh my gosh, must accept and give all the money!" pile. You might consider applying to more programs to compensate.
EDIT: As Ben Webster pointed out, "Apply to more programs" is incomplete. You might consider applying to more high quality programs.