I don't believe you can properly evaluate a deer's potential unless you observe that same deer for a few consecutive years. It is just too difficult to figure out why a rack looks a certain way, with just one look at a deer.

Most bucks will have the potential to develop very nice racks. One's potential might be a 140 class buck while another's is in the 200 inch range. To me, they are both trophies. I'd prefer to see all bucks reach their potential unless it is an obvious genetic problem. On managed lands, it's a bit easier to pick out these cull bucks. From your description, I would say that these bucks have a lot of potential and you should let them grow, if you are after larger racked bucks.