That really depends. A landlord with 10-20 units and a management company may not have to work, but they have a lot more in common with the working class than they do the Waltons or Buffets in the owner class.
In terms of net worth, sure. 10-20 million dollars is pretty much a billion dollars away from being a billion. But in terms of paying taxes? Unearned income pays the least in taxes. Psychology? It depends. If they have big mortgages and have to do a lot of the work themselves and be really careful with money? Then they might not feel rich (and may actually have a low net worth). If they own it all outright and have a employees take care of it all? That’s pretty different.