I really think it doesn't matter to deer. Why?
#1 I've watched deer many times near my stand during gun season, when one of my neighbors shot fairly close. The only reaction I've ever seen is for them to stop what they are doing and look around.
#2 There's a conservation department shooting range, near an area that I usually hunt a couple of times a season. It gets nearly constant use right up and through the rifle season here in MO. It is surrounded by small pine trees at the edges all the way around the shooting area. You can go there any time from about the 2nd week of october and find rubs all over those pine trees. In fact, this year I took an early trip to beat the crowds that will eventually build up there and what did I see picking in the mowed down clover patches? A nice 8 point buck.
The fact is, if whitetail weren't the marvel of adaptation that they are, we would have hunted them extinct long before now.
I'd say check your gun. I'd rather have the peace of mind knowing my gun shoots true than to have that nagging doubt in the back of my mind.
GH