Why do fatty foods tend to go with acids?
Mayonnaise and hollandaise contain lemon juice, and fatty fish tends to be served with lemon. Barbecue pork and pork belly are sometimes served with a vinegar-based sauce. I've also heard that fatty foods should be paired with acidic wines.
Is there something about the way we taste fat or the way we digest it that makes us crave sourness?
In regards to the general question, our brains are wired to like fats; they are compact sources of energy. However, too much fat in a given dish can feel unpleasant in the mouth, and acid washes that away (in terms of perception, the same way that adding sugars changes the perception of how acidic something is). I believe (by which I mean this is what I think, with no evidentiary basis whatsoever) this is a learned response, but I could very well be wrong.
The pairing of acids with fish, however, does have a scientific basis. According to the bible:
"Acid conditions cause heavy-smelling aldehydes, which accentuate the fishiness of TMA, to react with water molecules and become nonvolatile, so that lighter alcohols dominate the aroma."
(2004 edition, p. 238, hardcover)