What is your routine between the shots? Swabbing, wet, dry, none, etc? How long between the shots?
Just a thought, but I want to eliminate cleaning routine & barrel heating up as possible factors. Especially when the first shot was where you wanted it.