If I couldn't find an accurate load with less than 120 gr of BH 209, I would not use it for several reasons: gets too expensive, and makes me flinch too much

I would keep the powder charge lower and try a different bullet/ sabot combination. My elk load is 90-95 gr of powder depending on which bullet I am using, so shouldn't be a problem on dropping down to 80 or so.

That said, I don't ever shoot sabots, so they may require more speed to get accuracy, but that doesn't jive with the fact that you were getting better accuracy with lower velocity (pyrodex). 90 gr of BH 209 should be slightly faster shooting than 100 gr of pyrodex