If the people that were told the building was going to collapse had zero proof, but trusted what the builder said, are they jerks for trying to save people from the building?
I think any reasonable person would say no. In fact, any reasonable person would say that they were selfish and being jerks for NOT telling others about the impending collapse.
We even have laws about this in some places.