I have been self employed most of my life so for me it's not an issue. I really enjoy my hunting. I would quit a job that would not allow me to hunt. It may sound strange to some people. I'll bet if you go to an old folks home and ask them about their regrets none of them will say"I wish I would have worked more".
A lot of them would say "I wish I would have done more of the things I enjoy"
I don't want to have regrets later in life.