If your work is niche, then you can get a higher pay rate and work less hours because you are in greater demand.
While it might work that way currently, I dont think its really desirable to have some people work way less than others based on skill. I dont think me being a software developer should result in me having more free time than the health care worker.
And beyond that, overspecialisation leads to siloing of expertise and common knowledge. People with specialised knowledge having the freedom to reskill, or being able to spend some time doing more meditative, menial work and mixing with people with all different types of knowledge and skill will enrich their expert knowledge, not detract from it. Cross-pollination of knowledge is a real thing.
