This sounds like your barrel may be warping as it heats, since, being free-floating, it is not " bouncing off" the wood of the forend upon firing. This implies that the bore is slightly off-center, so when the bore steel expands from heating, the muzzle is pointed slightly off the line it was on when the shot was fired. To test this, try waiting five minutes between each shot. If you fire five shots in a row with 20 seconds between them, does this walking continue?

Your barrel might be O.K., and just need dampening by some upward pressure in the tip of the forend. Try putting a thickness or two of a business card under the barrel at the forend tip to give you between 10-20 pounds of upward pressure, and see if this helps.