Are you shooting with both eyes open, or one eye shut?

It may also be an issue with where you are focusing... are you looking at your pins or your target.

I'm blessed with very exceptional eyesight, so I can't exactly feel your pain, but the way I shoot is to squint one eye and center the peep around the sight housing... then I open both eyes and never take my focus off the target after that. The peep just makes sure my head is in the same place each shot. I don't even notice it.