Thats true if your peep and sight are not setup correctly.
When I setup my peep/sight window I do so by drawing my bow back with my eyes closed and my head straight. When I settle into my anchor I open my eyes and if everything doesn't line up I adjust the peep and/or sight (pinguard) until it's perfect. I do see alot of people who adjust their head instead of their equipment.