I set mine up the same way you do.
The incoming gobbler tends to stare in the same direction that the hen is facing and I don't want that gobbler picking me out....besides.........he really wants to get to the rear of the hen anyways.....I'm sure you catch my drift.
On the other hand I will set up a jake decoy facing me since the incoming gobbler will more than likely want to get in its face and challenge it.