I don’t think that any single score is going to make everyone happy.
Maybe if there are multiple user-scoring systems run by various sources, and I can choose which score I want to use as a metric.
Like, I think that the Marxist-Leninist crowd on some of the left-wing instances is bonkers, but I imagine that they’d say the same thing about me or other people who subscribe to mainstream economics in general. You’re not going to find a Single Source of Truth on that matter.
Eh, that was the whole point. Do not leave moderation to other people or at least make that easy.
It should be relative
Which means that the score of anything would be derived from 1) what you directly set, 2) what another user sets, modified by what you set for that user, 3) what a user sets, modified by what is set for him by another user, which has a value set by you attributed …
One can even make a logic where you see high score for things disliked by people you dislike.
There is some computative difficulty, but nothing big for our times.