Of course riding at 10kmh without a helmet is crazy, I mean you put a helmet on to climb a ladder don't you? And surely on a Saturday night when risks are really high, you wear a helmet as a pedestrian? ,and just like all these super safety conscious guys, you wear body armour when riding XC?
Helmets are magic, anytime you damage one they save you from being a vegetable in a hospital ward, imagine the tens of thousands of mountain bikers who have been saved from serious head injury filling our hospitals up if it weren't for magi-helmets.
I have to say though, I can't see the bother for removing an XC type lid. Removing it just means you have to carry it, which while I've done climbing a long hill on a hot day, isn't exactly the funnest thing ever.
IF it's a full face and you are riding to the place and it's hot and you can get it on your backpack - well, that's easier to understand - I wouldn't wear one of those hot boxes for more than 10 minutes. Ultimately, you are probably safer with a full face on the down bits and carrying it up, than an XC both down and up.