My biggest buck was taken while still hunting, minutes after putting out a smoke. I feel that the danger comes from the movement and sighs of pleasure rather than the smell.
I've been told that the smell makes them curious, I'm not sure I buy that, but unless they have been shot at by a smoker that they could smell at the time I don't think a deer will immediately associate the smell with danger.

"guns aren't for killing people, guns are for killing dangerous and delicious animals"
- Homer Simpson