Sounds like you are in good shape, the fraying is normal...

In my .54 the first shot out of a clean barrel is with a .530
ball and a .018 patch...If I have time for a quick swab for the
2nd shot, I load another .018 patched ball from my loading block...
If pressed for time or needing a quick 2nd shot, I also have 2 balls
with .015 patching in the block that give decent hunting accuracy...