You can, actually. But aside from that admins are able to defederate because it’s easier to filter out trolls and nazis and what have you. (Eg lots of instances defederating from Lemmygrad)
That’s not how federation works though; any community can defederate with another one. On different servers however, everything is still accessible even without logging in.