No you dont really but Marian and the others yell at me to do so because apparently some guy in the castle will attack you if you don't have it on and its not worth the trouble. You'd have to ask them about it, but plenty of ppl don't wear thier uniform. It's so itchy and uncomfortable anyway.

But if you're in the dorms, just wait there for a little bit and I'll send my little rain swallow to come help you! He's a little flaming blue bird, can't miss him.

[And true to Yamamotos words just a little while later a tiny blue swallow would be zipping in and out of the dorms before lighting down on a bed near Killua and chirping at him.]