RE: This scenario--"the deer won't come within 40 yards of me for some reason"....
I have smoked on stand but very sparingly. I always have a scent killer with me and only when the wind favors doing it and throw the butts in a sealed container(35mm film container or empty water bottle). If I am in an area thick with deer, i don't smoke. One stand I have been hunting deer come from all directions and I am hunting on the ground. I shot an 8 point this year that came from downwind of me. I doubt that would have happened if I was smoking.
I also practice very good scent control, so it make me not want to smoke, but If I am sitting all day, I will have to have a smoke and will do it during slow times. Some of my stands are on the high side of a ridge, where the downwind side is in a valley. This is bonus for smokers and for when the bottom end has bad chilli/beer breath. The scent line may be 50 feet above the deer, so i don't worry to much about it when hutning these stands.
Do I reccomend it? No, Do I do it? Sometimes. Have I killed deer after having a cigarette? Yes.
Other reasons why deer dont come close? The clothes or camo the person is wearing. If the hunter is using old camo that is faded, it sticks out like a sore thumb. Some of the faded camos almost have a white sheen to them when the sun is rising or setting. Same with clothing made in China. They use UV brighteners in their fabric to make it look better on the store shelves. For this you need a UV killer spray.
The last reason is. A lot of people are novices. They move to fast or to often. They wear bad clothing, smell like city folks, and often times do not set up thier stand with proper cover.