If nobody else is bound to shot him then better wait and see.
It would also be wise to let any deer with big size genes to grow to full maturity and harvest the lesser ones so most of the deer will be bigger on that area.
It will only work however if nobody else is shooting the biggest deer in the area.
Kind of leaving the biggest bull to breed the cows to get big offsprings.