The walkers show a rudimentary form of pack behavior, in that one walker spotting prey will frequently alert other walkers in the area to the prey's presence.
There have been a couple of scenes, for example, where there were walkers wandering aimlessly in an area, and a human got close enough for one walker to recognize them, at which point all of the walkers seemed to start noticing them.
Thus, the important camouflage benefit from the "tame" walkers is their lack of reaction.
If I recall correctly, Michonne even explains this in the comics, stating that after awhile, her two pets realized they couldn't bite her, so they stopped trying to attack her. They were an effective disguise because other walkers in the area saw that they weren't attacking, so didn't think there was any reason to suspect Michonne was human.
In the episode where they covered themselves in the walker organs to hide their scent, some of the walkers acted as if they were "suspicious", sniffing them and then ignoring them. This suggests they identify prey by a combination of scent, sound and sight, but that at least the visual and olfactory senses are of limited effectiveness, even relatively close-up.
The visual cues they look for probably include flight responses as a primary indicator. If it's running, it's prey.If it's making odd sounds, it's probably prey, and should be checked closer. If it smells like human, it probably is prey, but if it smells like human and walker, it could just be a walker that ate recently.
In short, the pets work because they mask the scent, and give other walkers in the area reason to think there is no prey around, through lack of visual cues (i.e. the pets, who are clearly walkers, aren't attacking, and the figure leading the pets isn't behaving like prey).