Some guns like my old 30-06 are very sensitive to barrel heating. The first shot would be ok, then after 4 rounds under 5 minutes it would wander all over the target six to eight inches sometimes.
This indicates either a bedding problem or a barrel with residual stresses present.