Shoot whatever makes you happy, as long as it's legal. Honestly, if anything you're doing is making hunting less fun, you're probably in the sport for the wrong reason, and would be much happier in some other "competitive" sport.
I shoot the first adult deer I see each year for meat, then wait for mature bucks, then thin the does out in the late rifle, blackpowder, and bow seasons. That's what makes ME happy. And that's the only reason I'm out there, to find some happiness in nature. I put my standards at a point where I usually see only a handful of "shooters" each year from the tree, which adds a challenge, but doesn't seem impossible to reach my goals. Some years I get the big one, some I don't, but I shoot a few deer every year, and love every minute of it.



"In heaven, even the fish have antlers"