If your patch was being destroyed after shooting, I would be concerned with the patches and the lube. Are these old patches? Were they lubed and then you lubed them again? I would try some different kind of lube.

If you have no other lube around the house, but have clean cotton patches try this... Put the patch in your mouth and get it damp with spit. Load a spit patch with the ball. Also if it starts to throw the balls like it did, try swabbing between shots with pure alcohol. Lightly apply alcohol, swab the bore and then dry patch. If you have doubts about the bore being damp, pop a cap.

Try that and see what happens. Although I have seen some rifles at 50 yard start to spray. You might also want to kick the powder charge up to 85 or 90 grains of powder.