I think if you look through pictures you'll find quite a bit of difference, but I suspect much of it is illusory. Specifically, you'll almost certainly find more pictures of really spectacular sunsets than sunrises.
I'm not at all convinced that this is because an average sunset is more spectacular though -- it's a simple matter of the number of pictures taken. Far fewer people are awake and taking pictures at sunrise. Given a larger pool from which to select, the pictures we see of sunsets are naturally better.
That's not, however, to say there can't be any difference at all though. Most of the physical reasons (I can think of) probably do favor sunsets as well. During the day, the sun is heating the air and the ground. This leads to air movement that tends to pick up particulates (e.g., dust, smoke, etc.) and keep it in the air. It also leads to water evaporating (relatively) quickly from anything like lakes and rivers in the area. In addition, quite a few things that people do (e.g., driving) tends to happen primarily during the day, so it "pushes" more pollution into the air during the day. Thus, at the end of the day, the light is shining through air that carries more particles, which almost inevitably does more to diffuse the light.
During the night, there's a lot less pollution being put into the air, and the lack of heating means the air tends to be calmer as well, so by morning many of those particulates will have simply fallen out of the air, and the humidity either formed into clouds or precipitated. The clearer air does less to diffuse the light, so the sunrise isn't as spectacular.
Although I can't provide any scientific support for it, my first guess would be that the selection bias is probably the largest factor though. Sunset pictures we see are likely to be more colorful, simply because we catch/see a lot more of them, and are a lot more likely to just throw one out if it isn't really spectacular.
Bottom line: yes, you can probably guess, and you'll probably be right a little more often than you're wrong -- but unless you recognize the terrain or something like that, it's essentially impossible to know with any real certainty.
