Your long and narrow (250/40) may need magnum powder charges at 150+. If not, then keyholing might occur.
That was my experience when I ran PRBullet's 250/40 a few years ago, using 85gr GoexFFF. Mine was a boat-tail design in a 50-cal barrel and could have contributed to it going sideways.
If yours is non-boat-tail and inside a 45-cal barrel, you may see better results at longer distances. If you experience keyholes, try using bore-butter in the sabot-bottom cup, or try a fiber wad.