On the user profile you can see which posts a user has upvoted. But if you could see it directly on a particular post, it would be pretty helpful, I guess.

Upvotes and downvotes could simply balance each other out. Since upvotes and downvotes are public, it should be possible to check the user profiles in question. But of course, besides surely being technically difficult, that would require that it’s actually the same person. This seems impossible to prove, as long as the same username can be registered multiple times on different instances.

However, I think it would be possible to keep a central database containing only the information which username has already been registered within the Fediverse - a bit like domain registrars. When a new user joins, the operators of an instance could look up whether the desired username is already occupied on another instance. This would certainly mean losing some autonomy, since the instances would no longer have sovereignty over available usernames. But I think it would be beneficial overall if usernames were only assigned once within the Fediverse.
For example, when it comes to counting upvotes and downvotes. But also to protect users from being discredited: I’m afraid that with the status quo it is quite easy to impersonate another user, since you can register the same username on another instance and do whatever you please with it. But that’s a completely different question, which I fear will become more relevant the more popular the Fediverse becomes (unfortunately, not only users with good intentions will join).

But please don’t get me wrong: I find the decentralized open source culture of the Fediverse extremely desirable - it is, in a way, a return to the utopia of the early Internet. I am very happy to be here and to witness that exchange among people is indeed possible without the influence of major corporations like Meta or reddit and all their buisiness schemes.

I just think it’s important to have a reasonably meaningful “random Internet points” system, whether it’s called karma or something else. I think these points are (unfortunately) the central motivation for many users to post content, which is probably why they play an essential role in the mass appeal of any social media plattform.