They can, but they tend to be more mold-resistant than comparable plant matter (and from what I’ve read some varietals won’t mold, but I’m not an expert so I can’t verify or deny that claim), assumedly because there is less viable material for it to eat. A house made of wood is vastly more condusive to mold growth than a house made of mushroom would be.