I’m not at all sure that is actually the main reason why we have separate male and female spaces.
But in any case, suppose that one in a thousand men is a convicted sex offender. That wouldn’t justify treating the other 999 with suspicion simply because they happen to be the same sex as the sex offender. That would clearly be sexist.
If one in a thousand black people were sex offenders, you wouldn’t try to argue that makes it OK to treat all black people with suspicion. Or would you?