They tolerate the human scent that they are used too. I think if they smell joe neighbor out in his yard while they are in the woods, they don't care. If they came across a person's scent out in thier woods where they don't expect it...high alert.
I also wonder if they don't pay attention to the farmers scent becuase they smell him every day. Put another persons(hunter) scent out there and they become suspicious.