If I am not mistaken the tradeoff is losing add-ons but being able to install other services.
So… what is your experience? Are add-ons useful/common for your use case?
HA OS is the way to go.
You don’t want to have to think about it. HA OS just works. You set it up and let it run.
There’s no sense in trying to kerfuffle other things into it. You don’t want to do too much on the Pi anyway because it’ll lower the responsiveness of Home Assistant slightly. If you want a server that does things, buy a separate NAS and run it alongside HA OS.