Eldeguello explained the higher impact from the "duds" exactly right.

But I didn't see anyone address the higher velocity of the heavier bullets, that you attribute to the sloppy bore fit.

I don't beleive it is the sloppy fit that allows them to gain more velocity. If anything, a sloppier fit would result in less pressure generated, and less velocity. BUT other factors are also at work. Because you did get higher velocity I'm sure the heavier bullets are upsetting properly and sealing the bore, allowing perhaps even a more efficient burn of the charge resulting in higher velocity.

For every barrel/charge/projectile combination, there will an optimal load for velocity and powder burn. You may have hit on it with your load.

Of course, accuracy is another matter, and the highest velocity may not yield best accuracy (I find it usually does not).