You can leave it loaded as long as you like. Of course you take a chance of the powder charge getting contaminated after a period of time.

As for temperature differences, try and keep them to a minimum. If you hunt where it is cold, do not bring the rifle into a warm house or car. The barrel will condense moisture and that in turn will kill or damage the charge. Leave it in the garage with the muzzle end down on some soft cloth so that if it does condense any moisture it will go away from the charge.

I personally shoot the rifle off every night and start the next day on a clean barrel. In your case, you could pull the charge through the breech plug and wipe the barrel clean and then reload it the next day if you wished.