Lemoyne, The barrel was clean - I was just taking precaution to remove any excess bore butter before loading so it wouldn't comtaminate the powder.And the tool your are discribing is called a worm.
Cory- I got the patch out with a smaller bore brush. I just put it down the bore and spun it around an let the bristles catch it. I don't know why the worm didn't grab it.