If you are right handed you could be grabbing the riser on release causing the bow to torque right.
Best thing to do is get some one to watch you shoot.