Check your manual and make sure there are not conical weight restrictions on the rifle. Some of them have a 400 grains weight. I always buy maxiball in bulk through
http://www.blackjackhill.com/ and then lube them myself.
You can also shoot roundball. .490 roundball and a patch should go down the barrel. Load a light load of 50 grains of loose powder and I think they might shoot better then you think. They will also knock any ruff spots off the barrel walls. I have a little Knight Wolverine and with 50-70 grains of Goex 2f and a patched roundball this little rifle is accurate enough to small game hunt with.
Since you looked down there and could see nothing maybe it is just the scope. On my Stainless, when I would swab the barrel, the patch would come back torn. After I J-B Bore Pasted it, the patches come out fine now.