In the glaciated regions of the northern tier, you couldn't grow an oak tree even if you popped an acorn up your booboo and incubated it.
I don't think that deer can hurt the habitat in pole timber, but is it fair and humane to try to range too many deer in that scenario to begin with? That is not the best thing as far as herd health. When you carry too many deer in that condition, they will gather in heavy concentrations in wintering areas and malnutrition and disease from close contact will emerge. That is not a very good life for a deer IMO.