Where would they be able to go, if they had to avoid all places they have seen a human in the past?

Deer really aren't all that bright. They have highly developed senses and survival instincts. They pattern hunters as much as hunters pattern them. Deer will adjust their movements to avoid stand locations, if the stand is overhunted especially in a bad wind direction. Deer will not avoid an area because they saw a person there one time.