I dont think you are being to anal when it comes to accuracy.
I try to maintain groups of 1 inch per 10 yards.
Plus I dont think the convience of a lighter bow to be worth the loss of accuracy.
I also shoot the Patriot and yes it is a little on the heavy side but thats one of the reasons I like it so much.

We owe it to the animals we are hunting to be the best shot we can be and to make a clean and ethical shot.