My guess would be you are not using the same anchoring point. I have in the past caught myself doing this. My ancoring point is my kisser and the side of my nose. Where my problem was is I would catch my self touching the string to the end of my nose instead of the side of it. I was inconsistant with my ancoring point. My accuracy wouldn' t suffer alot but there defentily was a difference in where my groups hit the target.