I would say that the most likely candidate is canting the bow (leaning the bow to one side or another). This can and will result in some serious left to right misses. Get a level and put it on your sight. This will give you a perfect vertical hold when your eyes are playing tricks on you (shooting on hill side slopes are notorious for this). You will be amazed at how far it away from your percieved vertical actual vertical is when shooting on a hill.

The other thing mentioned is the 3rd axis being off. This is a condition where the body of the sight is not perpendicular to the travel of the string. It can also lead to some pretty significant left and right misses, however 3rd axis problems normally only show up when shooters are using a level.