Colorado did a state amendment by referendum to actually ban slavery, but when a prisoner went to court to stop his slavery, the judge said the people obviously did not mean to ban slavery when they voted to ban slavery.
“the voters did not intend to abolish the DOC inmate work program by virtue of passing Amendment A,” wrote Judge Sueanna P. Johnson
The text of Amendment A which the voters voted on was:
“Shall there be an amendment to the Colorado constitution that prohibits slavery and involuntary servitude as punishment for a crime and thereby prohibits slavery and involuntary servitude in all circumstances?”
I remember following this in a few states that had this type of thing on the ballot.
In general, I think referendum voting is more legitimate than representative voting, but there’s no accounting for the judiciary just deciding it doesn’t matter.