I believe that letting the buck go will help the deer population but only under 1 condition. The doe have to be reduced in many states to have a good ratio between them. That is what will bring a healther deer population.
It would also help during breeding. Instead of having the 1 1/2 year old doing most of the breeding eventually it will become where they will not.