This is my totally unprofessional opinion. My guess is you have fissures in your barrel. After a round goes down the barrel the temperature causes the barrel to flex thereby changing the POI.

Some folks spend good money having barrels chryo treated. My dad is a metalurgist/robotics engineer and he says the form of chryo treatment that is applied to barrels doesn't do much to the metal to be worth even trying.

Either know how the rifle hits on the first shot or get a new barrel.

Good Luck

Tom