The missing factor in the discussion so far is the ability of the projectile to grip the rifling at various velocities.

A cartridge rifle can have a twist as fast as 1:8 because a hard jacketed bullet that is slightly larger than bore size is swagged into the rifling and the bullet is so tight that it will grip the rifling and spin. Try to push a lead bullet too fast down a fast twist cartridge rifle bore and it will "strip the bore" (i.e., lead will be shaved from the bullet, filling the rifling with lead) and essentially shoot like a smoothbore.

In a muzzle loader the projectile must be the same size or smaller than bore size, so it's impossible to achieve the tight fit of a cartridge rifle. A patched ball shot at high velocity from a fast twist looses accuracy because the patch isn't be able to stand the pressure and is torn asunder.