Can Muslim men wear shalwar?

Question

I have a question regarding clothing of Men. What is the difference between wearing women shakwar and shakwar which wears by men. Can Muslim men wear shalwar that typical style of women but material is not silk. Because at the end both are same plain simple. If men wear that shakwar and too kameez it's hard to recognise by that it's women shalwar. Unless it's in different style rather than simple or dark color like red.