The minni ball was never made for accuracy. It was made for speed of fire. All it had to do was hit any part of the person being shot at, or the fellow next to them, and the weight of the bullet did the rest. They were shot out of slow twist rifles 1-60 or even slower or a smoothbore. The common powder charge was 60 grains of slow burning 2f black powder.
If you shoot them too fast, you blow the skirt and they spray all over the place. If its a conical you want to hunt out of that black mountain magnum.. try a maxi ball. I think you'd be much happier. Even a REAL conical might do better. I have molds for both a 50 and 54 (or did I sell them?) and never cast bullets because IMO it was a waste of lead.