I think you may have it figured out on your own. Bowarm moving, causing rest contact with the fletching. Try learning to shoot with a better follow-through. Putting it nicely, the absolutely best shots you ever make will be those where you don't see the arrow go. All you were doing was aiming.