If the deer now routinely run on your building location, you can do one of two things...

1) Don't build
2) Don't worry about it.

Sorry, just had to say it. With the surrounding environment and microhabitats you listed, I personally wouldn't worry.

If you have the $$ or the machinery, I'd suggest clearing some trails on your 100 acres. Either tractor-sized paths or merely walking paths about 3ft wide. The deer will use these (or a select few, wouldn't that be nice) and become accustomed to the new house within a few months.

S&R