There are several components to justice, and capital justice in particular that feed into why decapitation would be the preferred method.
Firstly, as noted by @Cooper, there is clearly a tradition in the wizarding world of decapitation. There is also a paucity of other options: poison is fickle at best, and may not necessarily kill the convicted. This is especially true if the convicted is not human: how poisons work and interact might be difficult to determine - and experimentation is considered far from humane. Hanging is quite difficult to carry out properly in humans, even more so with non-humans - to do humanely requires breaking the neck in the first fall. Obviously this is made difficult in non-humans, and especially flying non-humans.
Magical means of execution are perhaps even trickier: there is only one mentioned death spell (Avada Kedavra) and it is considered quite heinous to use.
Typically the role of a headsman is one that is meant to convey the deliberate intent of the execution. That is, the conviction has been duly deliberated upon and is just, and that while the sentence is not one lightly handed down (in Britain, there is the tradition of covering the head of the sentencing judge so that the sin of killing does not fall upon them) it is fair to assign it in that particular case. A headsman is meant only within the well-specified boundaries of the law, and by extension the notion of justice.
Therefore, it makes sense that the Ministry of Magic has an executioner: formality is a must if there is to be a sense that justice is being carried out. For that sense of formality, it helps to have someone whose title and role is to carry out such sentences.
Given the wide array of potentially convicted beings, the axe makes sense: the removal of the head will in all known cases kill the victim. Axes are well designed for that sort of chopping, given their distribution of weight and ease of use. But there is a deeper tradition that makes this form compelling.
Specifically, within the Celtic histories the head is considered to be the home of one's soul, the center of one's emotions and indeed self and the connection to the divine. The Celtic god Bran, mortally wounded, instructs his followers to strike off his head and carry it back to Britian. One myth has it buried in London, thereby blessing Britian to be safe from invasion (at least until King Arthur foolishly digs it up). The heads of great enemies are similarly either preserved or displayed as either trophies or warnings, and sometimes are said to have magical properties. This is all because beheading is considered the most honorable form of execution.
Connecting this tradition and it's ties to the metaphysical to the wizarding world is easy. The Ministry helps to protect and build it's status as a respectable institution by conferring execution sentences that imply honor to the convicted. Proper execution requires that they be sure they can humanely kill the convicted, and harken to respected traditions of separating the locus of life from this world. Indeed, there are scant other examples of means of execution used at all, for all these reasons.