I can wear a dress to work and nobody criticizes me. I’d be really sad if I had to wear pants all the time, they’re just not as comfortable. Let’s normalize pretty dresses for dudes!
I don’t have experience with a sarong, but a saree is basically the same thing.
The difference is in the shape and size of the piece of cloth. That’s how you can tell a saree, bedsheet and towel apart. There is also usually difference in material (but fine silk towels exist, as do coarse cotton sarees), patterns/weaves (but there are towels and sarees that share pretty similar patterns) and quality of materials used (but again, ridiculously high quality silk bedsheets are a thing). The real difference is the shape and size - sarees are always 5.5m x 1.15 m (‘standard’ 6-yards), or 8.2m x 1.15m (9-yards, worn only on special occasions now, and only in a few specific regions).
In a pinch, a saree works as a towel or a bed sheet or a cover sheet of any sort, really. However, good luck getting a towel or bedsheet draped onto your body - you’ll look like you’re in a sack. They just don’t have the right shape!