RE: keep passing?
IMO, it depends. How much pressure does your hunting area recieve?
I can't control what the people on the bordering properties do, so I'll be darned if I am going to let a legal buck walk so some guy neighboring me that doesn't give a hoot about QDM can shoot the thing. I see it like this...it's gonna be me or him, and it ain't gonna be him if I can help it.
Now, if your area recieves very little pressure, let him go. You'll enjoy watching him grow up and having a shot at him in 2 years as an 8 or 10 point biggun. The down side of that.......that strategy could bite you too. As buck age they grow more and more solitary. You may not ever see him again.
It's a gamble either way. Kinda darned if you do and darned if you don't. What it truely boils down to is you. Are you man enough to set your goals and stick with them, or are you going to wimp out and modify your goals to satisfy your urges?