Do you know what the fuel costs alone would be? You don't scramble all your fighters to swat away a couple of rebels. And indeed, almost the entirety of the rebel force was destroyed extremely quickly. You don't build a great empire by burning money in a TIE engine - you need thrift :P Since X-Wings and Y-Wings are generally considered superior (though much more expensive) to TIE Fighters, they presumably launched an overwhelming force.
Less tongue in cheek, you don't commit a disproportionate part of your defense force - that's just bad strategy. What if the attack was just a diversion? And think about it - would you really expect that an assault of twenty or so fighters was the main attack on a battlestation that measures tens of kilometers in diameter? The attack force was utterly insignificant, and if it weren't for Luke, it would have no chance whatsoever of succeeding in its objectives. Are you going to be the one to explain that you left the station vulnerable to another attack (remember, the station is huge - it takes a while to circle it; and the craft likely need a few hours in maintenance after a flight)?
In the first movie, we see that X-Wings generally fight in pairs; TIEs seem to be fighting in triplets. Given that TIEs are vastly faster than X-Wings, it sounds like these are perfectly fine odds despite some technical advantage on part of the X-Wings - especially in constrained conditions like this, I'd expect a 3:2 ratio would be quite enough. Y-Wings might actually be somewhat more of a bother, since they can shoot backwards - 3:1 might be more appropriate.
The important thing to consider here is that fighters get in each other's way. The Death Star was emitting heavy ECM, which was affecting imperials just as much as the rebels. Under these conditions, they had to fight pretty much like WWII planes - and as we've seen in our world, contrary to what you might expect, a force too large can get in its own way so much that they perform worse than a small force. You can't just put more craft in the fight the way you can with naval ships. You need manual target detection, organise yourself in a massive 3D empty space with garbled communication and essentially useless navigation... there's no simple way to say "everybody focus on this fighter"; and even if you did so, you'd just get friendly casualties from collisions and friendly fire.
In short, despite what people tend to claim, there's no guarantee that launching more fighters would help. It could even make everything worse (remember, we're talking prediction here - as the commander, you didn't know that the result might be a descruction of the whole station; nobody really seriously believe that could happen).
I couldn't find any reference to an actual number of TIEs launched. However, given the combat we see (and hear about) in the first movie, I'd suspect a ratio of about 3:2 to 3:1 to rebel fighters. Of course, this is just a guess - but I suspect you're not going to get a definitive answer. Neither the movies nor the novelisations specify the actual number, and even if a number was mentioned somewhere in EU, it would likely be an offhand comment like "Those are the guys who defended their 42 Bajillion Credit station with fifty fighters, right? :P"
