Until recently (when I finally set my NP) I was shooting good groups, just sometimes left, sometimes right, etc. I wasn't worried, as I was more interested in arrow flight and felt confident it would all come together when I was done tuning, sighting in, etc.
Read this article this morning that made a lot of sense and most likely explained alot of what was happening-
http://www.spot-hogg.com/newsletter_12.htm
Exact and repeatable hand placement on the bow is so critical to consistent accuracy. I've found that even with my NP dead on, while the left and rights are pretty much gone, I can get high fliers if I put too palm pressure on the riser.
I plan to try the glove as they recommend to assist the bow into settling where it wants, as generally I shoot with a bare hand, even in the woods. Just need to find a glove now. I'd hate to buy a new one, as I have a ton, but they're never around when you need them