BH will test the setup and tuning of both the arrow AND bow. As Paul said, if you are underspined, you don't stand a chance. If you are spined right to a touch on the stiff side, there is a good chance.

BH will change the FOC a little bit, but the thing that gets most BH off line are either the insert/bh are not alighned right, or the arrow is leaving the bow at an angle. With FP the fletching can make up for either, that's its job. With a fixed blade BH on the front, now you have the wings of the BH trying to keep the front of the arrow going in the direction its going. When its not straight you get air flow actually pushing on the blade to push the arrow further off line.

You say 4 inches or so at 20 yards? That's a lot and indicates a tuning issue of some sort, bow, arrow or both. You can "tweak" the rest SLIGHTLY towards the FP grouping, slightly as in 1/32 inch at a time. This fixes any leaving the bow straight issues.

Fletching also makes a HUGE difference with BH. Try a more offset fletch, or a different kind. I've found feathers to be alot better than vanes.