It's rude. And it's harassing. There is intent. If someone said to a woman in a skirt, "Hey, you going commando?", it wouldn't be "just rude".

Jokes/questions about bagpipes, those are just rude. But making enquiries about people's undergarments is (IMO) a step past the overly forward, and into intentionally something past rude.

That so many joke are based on it doesn't move it out of harassing, any more than rape jokes make directed comments about sexual assault only rude.