One thing you may consider doing is letting more family hunt but with strict rules etc...

ie absolutely no button bucks killed,
no bucks under 140"
under 3.5yo...and so on

if that's what you're holding out for.
personally if a family member takes a great buck I'm just about as excited as I would be if I took one myself, but when someone takes a dink and we have standards/goals that gets me mad.

so my thing would probably be, sure you can hunt, but you have to follow the rules, or that'll be the last time you can hunt...