It really depends on if you think you can get to your stand without spooking the deer away. I am lucky enough that I can hunt only 15 minutes from the house, and as a rule the deer dont come out until dusk. If I have only 1/2 hr to hunt, I will try to be there in the right conditions. If you dont go out at all, you never know.
At the same time, I am very careful never to knowingly allow deer see me enter or leave the tree stand, because (whether actual or not) I believe once I am spotted, my stand is toast with the deer being more alert to my possible presence.