I have hunted for a few more years than I like to admit, and in that time I see the most deer right around 7am and 9am local time. Not much before or after that. And this was when I wasn' t smoking.
So, I usually will light up between 7:45 and 8:15. And then again after about 9:30. It never seemed to mess up my hunt. In fact, one year, I was almost ran over by a young button buck while sitting on the ground, smoking a cigarette. His eyes got as big as mine when he realized what I was.
But, there is a safety factor here. Make sure it' s really out before sticking the butt in your pocket.

That alone might cause you to fall out of your tree!!!