Perhaps some of the focus on big racks may be attributed to the abundance of deer these days. As some have already said, if just taking a meat animal was the objective, the hunt would be over by 8 AM opening day: what fun would that be? I believe it is definitely the case that hunting bucks with bigger racks is different than hunting does and/or hunting young bucks with small racks. These mature bucks offer a substantially higher order of challenge. I think you are probably going to be disappointed watching hunting shows on TV, for a variety of reasons.
I don't happen to define my goals in terms of rack size. I will say I prefer to hunt on an area without food plots and without feeders. I like to hunt from improvised ground blinds not from tree stands, not from artificial or constructed blinds. I'm generally as happy taking a doe as a buck. When I take a buck, I save the horns and mount them myself, no matter what the size, along with a commemorative brass plate describing the particulars of the hunt. Sometimes -- even more often than not -- I take my deer on opening morning, but not always. I guess if it was 80% probability that I would finish opening morning I might look for a different challenge, but I haven't gotten there yet. I have begun elk hunting near tree line in Colorado, and I can see myself dropping deer hunting by preference for the mountain elk hunting.