My two piece is always on, and I can't say that I've ever missed a shot because of "less accuracy" that is theoretically caused with it on.
And to say it causes more vibration is also incorrect. Actually more mass/wieght on a bow will tame the vibaration quicker than less mass/weight.
I'd bet that 8 blind folded guys out of ten wouldn't be able to tell if a bow had a quiver and 5 arrows attached, or not!