The employed are punished by having to do what they do not love. The self-employed are punished by the opposite.
It is only if the primary or only reason you do what you do is to make money that you will envy every random person who made or makes a lot of money (or money that exceeds what you made or make).
If self-employment is the opposite of employed-by-other, than a job is outsourcing your freedom (time and talent) to a boss.
Self-employed people work where they live. Entrepreneurs live where they work.
Employees leave where they work. Employers live where they work.