The reason O'Brien is identified as "chief" actually depends on which episode you're watching. His military rank history is a bit unclear throughout the show, but at any given point, one of two possibilities always holds for that form of address:
Initially, O'Brien appears to hold the rank of lieutenant (he has the same uniform as other lieutenants and he's addressed that way several times.) However, his specific job duty is "transporter chief". As such, it's acceptable to identify him as "chief" informally.
As some point late in The Next Generation, and then for the entirety of Deep Space Nine, O'Brien is identified as a senior chief petty officer. In this case, his rank allows him to be identified as "chief". He was also promoted to Chief of Operations, an NCO position, so it was also proper to identify his job title as "chief".
There's a bit of a flaw in this progression: lieutenant is a commissioned officer rank, while petty officer is not. There's no reason given why he would have lost his commission (did he resign it, was it stripped, etc.?), especially given that he continued to be assigned more responsibilities as he went along.
Regardless of his rank, though, there was always some legitimate reason for other crewmembers to address him as "chief".