There are several things that determine if the deer you saw will ever become more than "basket bucks". First thing is genetics. If the buck doesn't have the genetics for a impressive rack, it doesn't matter how much time and nutrition you give him, he just will never delevop into a "trophy" buck. Now, the question is what a throphy is to one person, may not be to another. My friend and I let alot of smaller bucks walk but allow younger hunters to harvest them. We do try to harvest the ones that don't look as though they have the genetics we are looking to keep first. We basically call these "cull bucks".
Now, if the buck has the "right stuff", then he needs the proper nutrition. That means that the number of deer can exceed the carrying capacity of your land. You can also suppliment the feed with feeders or food plots. We have seen a huge increase in the size of our does and fawns since we began our feed program. But trust me, it isn't cheap!
Lastly, the buck needs to mature to develop his full rack potential. The earlier years, alot of the nutrition goes to building his body as well as the rack, but around age 3 - 4, the body matures for the most part, so more nutrition can be focused on the developing rack. Of course like anything else, a buck will reach his prime and then begin a down hill slide. I have a huge nine point on video that we have watched for a couple of years. I think he peaked last year and is now starting on the decline. He is a huge deer with a heavy gut, a gray face and some extreme mass in his rack.
Bottom line is that you have to find a balance is the equation. if you hunt public lands or land with an over abundnace of deer, or with extreme hunting pressure the chances of a buck getting truely large are really decreased.
Anyway, that's my .02 worth!