RE: am i hurting my deer population
It sounds like you have a good site. Might want to also get permission to hunt the areas surrounding your new house. Deer stay in the area they are born, won't go to far unless forced to. I think many poeple on this forunm would love to build a house in the woods.
Consider putting in food plots. What a way to help maintain your deer population.
Good luck