As to your accuracy issues I think your dead on with your observations. When you can not get consistency with each shot respectivly, then you can not really expect accuracy. Just the fact they one loads easy, the next hard, etc I have found plays havoc with your accuracy and group size.

The first thing I would do about the rifle is, get some J-B Bore Paste and scrub that barrel paying special attention to the breech area. I would want to make sure that area is smooth and free of any burrs or anything that might be causing the problem.

Second I would suggest going to a cleaner burning powder like APP, Pinnacle, Black Mag3, Shockey Gold, where the crud problems are not as dominant.

Last would be to change sabots to something like the MMP HP-24, Harvester Crushed Rib, Super Glide, or any of the thinner sabots made to combat this hard loading problem.

Keep up the good swabbing routine. Also you might want to start swabbing with a separate range rod which has a bore brush attached to it. Wrap a patch around the bristles, dampen it with a good cleaner like Rusty Duck or Birchwood Casey #77 and swab with that. It sounds like you have excessive crud ring issues and until you can find out the reason for them, this would be the route I would take in order to keep shooting.