Mild rain doesn't seem to bother them. I've hunted in it before and seen deer. They only seem to hold tight when it's really bad weather - hard rain, wind, etc.

The question shouldn't be whether it bothers them, it should be whether it bothers you.