Fluffy's chamber on the third floor didn't have any special lock. It could easily be unlocked using Alohomora.
Alohomora was a basic spell which even first years could use and that's how our heroes got into the Fluffy's chamber.
Instead of instructing students to not go into forbidden area on third floor, why wasn't the door simply locked with anti-Alohomora charm? It could also keep the thieves away.
Talking about feasibility of anti-Alohomora charm, in the quest to Philosopher's Stone, the trio found Alohomora not working on a door whose key was flying around. It means that doors can be locked in such way that Alohomora couldn't work.
Why wasn't the first door locked such way? Dumbledore himself could carry its key in his pocket.