Id unload it if it were me. The temp change from going outside to inside is going to make the barrel sweat, which could foul your charge.
Would it be fine? Maybe but do you really want to take the chance if the buck of a lifetime walks in front of you and your gun misfires?