I often leave mine loaded until a buck comes by. If you use a muzzle sealer like muzzle mitts or thumb cots it closes off the muzzle and I made a rubber copy of a 209 to seal the breach when not hunting, I shot the best buck I ever got in a downpour with a carefully sealed up Hawken and a round ball. with a good gun careful preparation the weather or time as long as its reasonable does not seem to be a difficulty to me.
The only thing I would add to that is that if I was using a hygroscopic powder [attracts moisture] like APP made powders or 777 I would be a lot more concerned.