The fastest way to load roundball is off a loading block. You make a loading block and then it is a matter of dumping a speed charge, setting the block over the muzzle and with the short starter, driving off the block.

Using hardwood like oak or maple, in any design and number you like, drill 1/2 inch holes. You then lay the patch over the hole and push the roundball against the patch into the block. After you dump the powder, lay the block over the muzzle and drive the ball through the loading block.

I personally carry an oiled patch strip of cloth, and some ball. After I dump the powder, I set the ball just like I would do on the target range. Speed really means very little to me.