First of all, I've not read the book I'm only going off the film.
In the Prisoner of Azkaban, Buckbeak is sentenced to death with the sentence to be carried out by an executioner with a rather large axe. Given the seemingly large number of spells / curses / potions available to wizarding-types, why was such a crude and inhumane (not to mention messy) method of dispatching the animal used?
This is probably opinion based, but I'll take a shot:
Muggle "humane" methods of putting animals down arise from muggle medicine, repurposed for veterinary care.
Wizards have contempt for muggle medicine. Wizards would have no reason to develop pentobarbital, and thus would not have it available for use in veterinary care.
Edited to add: sorry, you also asked specifically about spells. The one spell most likely to be used here is Unforgivable, and perhaps there is no waiver to that for legal limit for putting animals down.
Short Answer:
It is probable that Rowling chose decapitation because:
1) It is, although messy, an almost instant and probably almost painless method of death, and thus not nearly as upsetting for child readers to think about as many other methods of killing lower animals or humans which muggles have used in history and which wizards might also use. I have seen a 20th century movie made for children in which a boy about ten years old was sentenced to be beheaded, for example.
2) Since beheading was considered the most high status and humane method of execution, in Western European culture for thousands of years, many readers, even child readers, would have already be familiar with some historic men, women, and children who were beheaded, and so might not think that the Wizard world was very cruel or harsh to grant a lower animal such a relatively humane and honorable form of death, and so might not think that the other wizards were just as bad as Death Eaters and wonder why should Harry try to save wizards from rule by Lord Voldemort.
3) Beheading presents a very dramatic instant transition between life and death. With many other methods of killing a person or animal it would be hard to tell when the victim was mortally wounded and would die anyway if the killing was stopped. With beheading, there is a sharp cut between when the victim is still alive & not yet harmed and could theoretically be saved and when the victim is dead. This makes beheading an obvious dramatic choice as a method of execution.
Long Answer:
In European culture beheading was been considered the most dignified method of execution for thousands of years, and thus was the usual method of execution for high ranking persons.
Humans have practiced capital punishment by beheading for millennia. The Narmer Palette (c. 3000 BCE) shows the first known depiction of decapitated corpses. The terms "capital offence", "capital crime", "capital punishment", derive from the Latin caput, "head", referring to the punishment for serious offences involving the forfeiture of the head; i.e. death by beheading.[9]
Some cultures, such as ancient Rome and Greece regarded decapitation as the most honorable form of death.[10] Many European nations continued to reserve the method only for nobles and royalty. In France, the French Revolution made it the only legal method of execution for all criminals regardless of class, one of the period's many symbolic changes. [10]
Others have regarded beheading as dishonorable and contemptuous, such as the Japanese troops who beheaded prisoners during World War II.[10] In recent times, it has become associated with terrorism.[10]
https://en.wikipedia.org/wiki/Decapitation#History1
If the headsman's axe or sword was sharp and his aim was precise, decapitation was quick and was presumed to be a relatively painless form of death. If the instrument was blunt or the executioner was clumsy, multiple strokes might be required to sever the head, resulting in a prolonged and more painful death. The person to be executed was therefore advised to give a gold coin to the headsman to ensure that he did his job with care.
https://en.wikipedia.org/wiki/Decapitation#Pain2
Decapitation is quickly fatal to humans and most animals. Unconsciousness occurs within 10 seconds without circulating oxygenated blood (brain ischemia). Cell death and irreversible brain damage occurs after 3–6 minutes with no oxygen, due to excitotoxicity. Some anecdotes suggest more extended persistence of human consciousness after decapitation,[16] but most doctors consider this unlikely and consider such accounts to be misapprehensions of reflexive twitching rather than deliberate movement, since deprivation of oxygen must cause nearly immediate coma and death ("[Consciousness is] probably lost within 2–3 seconds, due to a rapid fall of intracranial perfusion of blood").[17]
https://en.wikipedia.org/wiki/Decapitation#Physiology_of_death_by_decapitation
In ancient, medieval, and modern times, decapitation has been considered the least painful and most classy method of execution, and thus has been the usual method for killing or executing royalty, nobility, and other high ranking persons, in western or European civilization. Many famous men, women, and children in history have been killed or executed by being beheaded.
In ancient, medieval, and early modern times many or most people were often very cruel to wild animals and livestock, and only cared about the suffering of their own pets if they cared at all about the suffering of non human animals. The idea that people should avoid causing pain and suffering to animals became widespread comparatively recently.
And I get the impression that wizard society in Harry Potter has old fashioned attitudes compared to those common in European civilization at the time the stories were written. Wizards may have attitudes about some matters that were common in muggle society decades or centuries earlier.
So I sort of have the impression that if a bunch of wizards gathered to see Buckbeak be killed, some of them might be shocked to see the ax intended to be used and say: "What! Is this animal going to be beheaded like a noble human being executed? How dare they treat a lowly critter like it was a person, and a high status person to boot!"
Yes, it is my impression that a group of wizards might be upset by the idea of beheading a hippogriff for opposite reasons to those that the OP stated.
I remember watching the movie The Thief of Baghdad (1940) on television.
At the climax, the villain sentences the hero and heroine to be die "the death of the thousand cuts" tomorrow. When tomorrow came the executioner had a huge scimitar which he obviously planned to use to behead them, the death of only a single cut each. And even though I was rooting for the hero and heroine to survive, I felt disappointed that for some reason the advertised "death of the thousand cuts" was being replaced by the mere "death of a single cut".
So why did the script change the threatened "death of a thousand cuts" to "death by a single cut" which may have seemed like a let down to many in the audience?
Possibly in order to not seem too brutal and frightening to the audience and violate the motion picture production code.
I asked a question about a film for children where a little boy was sentenced to be beheaded at the Tower of London in 20th century Britain.
British kids's movie or TV show, teleporting into Tower of London3
Obviously the intention to execute a child by beheading is not considered to be too gruesome for a movie for children, while characters sentencing a child to be hanged, strangled, burned at the stake, or impaled might possibly be considered too gruesome.
And as I remember, the British television mini series I, Claudius (1976) had an episode, "Zeus, by Jove!", 8 November 1976, with a scene where the decapitated head of a child who looked about 12 years old was seen. I, Claudius wasn't a series for children, but it was on British television.
And possibly because beheading is more suspenseful and dramatic. One moment someone is totally healthy, and the next moment they are headless and doomed to death within seconds. With beheading, the clock is ticking and there is a clear deadline for any rescue attempts to beat.
But with slower and more painful methods of execution, the victim could be rescued after the execution had begun and might be still alive, but there might not be any way to be certain whether they would survive the damage or were already fatally injured. Only time might tell. That would not be very dramatic storytelling, even if the motion picture code allowed depiction of the more agonizing forms of execution.
So it is possible that Rowling was writing a scene in which a bunch of wizards prepared to kill Buckbeak by tying him down and beating him to death with clubs, or by stoning him, and decided that would not do. It would be too disturbing to her child readers to have an animal threatened with such a horrible death, and it would make typical, normal, ordinary non Death Eater wizards seem too cruel. Readers might think that if typical wizards are that cruel, what's the point in protecting them from the tyranny of Lord Voldemort.
And such a slow agonizing death might not be dramatic enough. Each wound or injury would be non lethal by itself, and only the cumulative effect of many such injuries would be fatal. So if the protagonists interrupted such a death and rescued the victim, they might not know for hours, days or longer whether they were in time or whether the victim was already fatally injured.
But beheading would be almost instant death and almost painless, and it would be an obvious deadline for the protagonists to beat if they wanted to save Buckbeak.
So Rowling may have decided on beheading as the threatened method of death to not seem too frightening to the readers, and to not make them despise all wizards, and because it gave a very clear and sharp deadline for the protagonists to beat, making it more suspenseful.
