It sounds like the barrel wasn't stress relieved properly. The 1st shot is always most important anyway so I suggest onlt shooting 1 shot at a time with plenty of time for the barrel to cool. This is far more common than people realize to some point. The acid test is to shoot a 5 shot group with only firing 1 time a day using same target. Keep your scope sighted in for 1st shot as that's what you need when hunting. The only proven solution that I have found to reduce or eliminate this (barrel "walking" as it heats up)is cyrogenically freezeing treatment that does relieve stress. I've sent dozens of rifles in and ain't ever had 1 yet that didn't shoot good. Of course most of these are best quality match barrel blanks turned down to sporter contour in custom rifles I've built.