RE: This scenario--"the deer won't come within 40 yards of me for some reason"....
I honestly don't think that smoking bothers deer any. Think about how many deer you will see and subsequently kill the year after a controlled burn? Fire and smoke are naturally occuring things. I'm not a smoker, and I am very scent control aware, but I just don't think it would bug them at all. Human odor of course is a big red flag.
One other reason that lots of hunters, and even those who do not smoke or anything, only see deer at 40 yards is because everytime you enter the woods you leave some scent behind. In the case of hanging a stand for example if you are not at least somewhat cautious about scent control while hanging the stand, when you'll likely be sweating and what not... deer can change their patterns to move around you very quickly. North American whitetail had a small aside during their show where their biologist said that almost everydeer that uses that piece of property frequently will know exactly where your stands are within 72 hours. Pays to be mobile.