The Rabbis are not against talking to women. They are against excessive talking to women. What is excessive?
The Mishna in Avot 1:5 says:
אַל תַּרְבֶּה שִׂיחָה עִם הָאִשָּׁה
Don't speak excessively to the woman
There is an interesting הָ/"the" in line. Ostensibly, it comes to mean "wife" rather than any woman, which would seem to go with the rest of the Mishna, but that הָ still isn't necessary for that.
I once heard an interesting diyuk on it. The Mishna is teaching us how to be a mentsch (this is Avot after-all). How does a mentsch talk to a woman?
A man flies from NYC to Boston for business. On the way out, he sits next to a man. The whole flight, the only thing he says to the man is "excuse me" when he needs to get out to go to the bathroom.
On the way back to NYC, he is sat next to a woman. This flight is very different, because he strikes up a conversation with her from the get go, and keeps it going whenever he can think of something new to say.
What's the difference? Both cases should be the same: he had nothing to say to the man and no reason to strike up any conversation, so he didn't. The same should have been true for the woman, but it's not so. The difference is, she is the woman.
The Mishna is explaining that this man isn't viewing her as a mentsch, a dignified human being. If he did, he wouldn't say anything to her unless he had a reason to do so, just like he does with men. However, he doesn't view her as a mentsch, he views her as "the woman". He's not talking to her, he's talking to "the woman". Therefore, he talks "excessively"; he finds extra, unnecessary things to say simply on account of her womanhood, and this is very silly, and not something a mentsch should be doing.
So, I think there is a misunderstanding in the premise of the question. Being a woman doesn't default preclude a man from talking to her, although there is definitely reason to not talk to her just because she is a woman.
May I recommend that you look up if it is mutar for a man to compliment a woman's looks, halachically, and if not ask why this is an exception?