sabotloader I also use a remmy and here the weather in ML season can be 50-60 to well below freezing, no changes in POI on the range in my T&E with a clean/foul barrel. I have done a few tests where I never swabbed and found some decline, particularily after the 2nd shot is fired when using 100gr range of Loose T7 (upto 100 grain pellets I can get 3 shot before this degrade shows on the target). In the field I do carry a small clean bag with me JIC, but thankfully never needed to worry about the second shot (at this point).
Like I said the only way is try and adopt the best practice for your situation. I believe whole heartily in practicing as I will hunt so I have pretty much tried everything imaginable to simulate what I may see on any given day in the field...thus I hunt clean barrels and don't give it a moment thought with this particular inline!