Justin, I don't ever shoot with my quiver on, it's always removed once I'm up in my stand.
This new Diamond Quik Disconnect quiver I've mentioned also has a belt accessory where the quiver can be carried on the hip--it's a pretty neat quiver.
I've got one for myself (4 arrow) and I'm quite pleased with it.