And just because a buck is small doesn't mean he carries good or bad genetics. Young deer die all the time. If that one buck is the focal point of your deer herd then you're one car/deer collision of being out of business.
When is this fascination with big antlers going to end? Whatever happened to hunting for meat? When did that become a bad thing?