If an Agent can possess anyone still connected to the Matrix, and, as we can see, they sense when someone is seen something weird (like the drunk beggar in the first movie), why didn't they possess Neo when he was in the car with Switch Apoc and Trinity?
I believe that's what Trinity meant when she said the gun was for their protection from him
Neo: what is this? (talking about the gun pointed to him)
Trinity: It is necessary, for our protection.
Neo: From what?
Trinity: From you!