What is the upside to letting these guys hunt land you've been "cultivating" for a while? I know for a fact it would be a big "NO" if I was in your shoes. You don't have any reason to feel obligated to grant these guys permission to hunt. If you think telling them "no" will make you looke like a "bad guy", then try another route. Something like, "My family decided a long time ago that we would keep it family only. Just too many of us to be bringing friends along - we thought it would cause too many problems." That oughta take the wind outta their sails and makes it look like the decision is out of your hands.
Personally, I'd just give 'em a big old-fashioned "Nope - sorry".