Robert, something is wrong somewhere. First you need to remember that the reason why you use a patch is that the patch goes down into the grooves of the rifle. That is what makes the ball twist as it gets shot down the barrel. So the thicker patch will engage the rifling twist, where a thin patch will not engage the rifling twist. And if the twist is not engaged then your ball will not spin down the barrel, it will roll down the barrel. So you might as well shoot a smoothbore because you will not be putting any twist on the ball.
"Hammering" the ball down the barrel is wrong. You are deforming the ball every time you touch it, so just imagine what hammering the ball has done to the uniformity of the ball??
Do you have a micrometer?? You need to check a few things out. For starters just see if the ball will "fall" into the barrel without a patch. then take your micrometer and actually measure the ball and the patch. And try to measure the bore with the micrometer. IMO you will probably find your problem right there. Something is wrong. A .530 ball with a .005 patch should load very easy, with very little resistance. So you need to measure where your resistance is at.
A dirty bore will also make it tough to load a projectile. But it sounds like you have other problems besides a dirty bore, but it would be worth checking. Tom.