RE: No matter how hard I try!!!!!
Odds are, it's either old Bore Butter/Wonderlube or very light rust. You should always use an alky patch after cleaning to remove water remnants -- then two double-sided dry patches -- then two single-sided patches of oil that says "rust preventing" on the front --- all with a bore brush. Make sure the oil can does not say "rust inhibiting".
Whatever is in the bore can be removed with JB Bore Paste... which will also remove any buildups of lead, carbon, copper, plastic & light rust. READ THE DIRECTIONS FIRST! Completely cover three 3" patches with JBs... alternating with gun oil & dry patches.... all with a bore brush. Then use enough gun oil patches until the patches come out clean. After the next time you swab the bore before heading out to the range, check for "brown again". If still there, do three more JB patches after shooting.
I always start slow with the JB & do less at one sitting -- rather than the entire project at one time. I'm still removing a ton of carbon from one of my rifles. I'm in no hurry cuz' it still shoots accurately. I'm figuring it will need twelve patches -- the equivalent to (4) times at the range.
I recommend no Bore Butter unless using it for roundball patching or lubing conicals. I recommend Birchwood-Casey Sheath for rust-preventing bore protection..... and don't forget to get an empty spray bottle & fill it with alky after using solvents to clean the bore.
Soapy water is for sidelocks -- power-flushing the powder channel in the bore. There's absolutely no need to put water in an inline bore with removable breechplug. Your best best is to avoid using it cuz' only (1) mistake of leaving traces of water in your rifling will cause rust unless you are using either alky to remove it or a gun oil that says "water-soluble/moisture displacing & rust preventing" like Birchwood-Casey Sheath or Ballistol.