RE: Whats stopping you?
The past 5 years I have done exactly what Duke is talking about above (good post BTW), and I have found that the very best bucks on my property that I can find through scouting, glassing, and talking with others in the area are bigger 2.5 year olds and nice 3.5 year olds. Are there odd sightings of a "big" buck that could be older, sure, but I'm not counting on hearsay for my hunting.
In the last 3 years I have shot bucks that would be in the top 10% for size on the properties that I hunt and have access to. So, to me, nothing is stopping me. I'm doing pretty damn well with what I've got.
If someone can come hunt with me on my properties and drag out P&Y bucks year after year I will bow to their hunting prowess. I don't believe that would happen though.
That doesn't mean that I am content either. I'm constantly looking for access to better properties that hold bigger bucks. When I get them I'll raise the bar to hunt the top 10% on those properties. Until then I'll keep working with what I've got.