The key point of this quote is that Dumbledore is not talking about protecting himself, but rather about protecting others, from Voldemort. This is one of the asymmetries between the two wizards, making Dumbledore's position quite rickety: Voldemort considers everyone else dispensable, so he ultimately has only himself to protect; Dumbledore, on the other hand, not only would not use others as pawns in his game, but often seems to feel responsible for the world at large as potential target of an attack. From this perspective, the magnitude of the 'protection' problem is not difficult to appreciate.
This quote appears to be ultimately about protecting Harry, who is a likely target (and considered "valuable", as we learn in HBP), but it is arguable that Dumbledore is quite sensitive to loss of innocent lives in general, so the principle behind it applies to other potential targets as well.
So the quotation is really not about beating Voldemort in combat: rather, it is about potential aggregate loss on either side.