The harm is only caused by those that would have bought the game if the avenue of piracy did not exist.
And the harm is in no way different from someone not buying the game because they aren’t interested in the game.
No, there is a difference, and my apologies for not responding to your statement about the mechanism.
The mechanism of harm caused by the first group (those that would have bought the game if the avenue of piracy did not exist), is that by choosing to pirate instead, they are removing their contribution to the profitability of the company and causing an increase in price to remain profitable. These increased prices cause undue burden only on those people purchasing the product.
There is no mechanism of harm caused by the second group (someone not buying the game because they aren’t interested in the game). In this case there would be no alternative action if the avenue of piracy did not exist because this group would still not purchase the game.