We know from the fact that Benjen periodically visited Winterfell that the sons of high lords were allowed to visit their families.
However, Jeor Mormont's home on Bear Island is (I think) a little closer to Castle Black than Winterfell is. Yet he seemingly never visited. From what I can tell, he was quite close with his sister, Maege and surely would have expressed some desire to meet her daughters, his nieces. It is possible that he visited Bear Island when Jorah was exiled and that is how he obtained Longclaw but still, if Officers were allowed to come and go I'm sure Jeor would have visited his sister and nieces.
Is there any particular reason for Jeor never visiting home?