If there'd stop being double-standards, there wouldn't be such an issue. Either everyone can wear something freely if they so choose, or no one can. That's the way it should be. Having religion as an excuse is too tricky, because anyone can just claim to be of a certain religious affiliation just so they don't have to follow a certain rule.