Therefore I pass a lot of deer waiting on a bigger buck etc. I am assuming from your post that you do not do this.
On the contrary! I pass on many animals every year for different reasons. Too small (fawns) , big doe with fawns at her side, young bucks early in the season (can fill freezer with doe). I can afford to be selective as we (Wi) have a very lengthy season and when all is said and done I know my freezer will be full. Others dont have the luxury of being as selective as I can be when trying to fill the freezer.
For everyone that has aspirations of killing a big buck then part of the reason that we hunt is for the challenge becasue if not we again would be shooting everything that walked by rather than waiting for a big one because if it wasn't more challenging or more rewarding, on a personal level, for us to kill a big buck, then there wouldn't be any more significance in killing a big buck then there would be in killing a fork horn.
Not necessarily a challenge , but rewarding YES! The big buck (trophy) thing is also a primal instincts trait or whatever you want to call it (testosterone?). It's our nature to be competetive against one another which is what I consider the trophy buck syndrome to be , competitiveness. We all have it. If I go out and kill the biggest sabre tooth tiger then I can pound my chest and have any woman of the tribe I want. I'm not setting out to do it for a challenge , I'm setting out to do it for bragging rights.
If you had a 150 class buck and a fork horn standing there, which would you shoot and why?
Needless to say I'd be pounding my chest!

Who wouldn't?