Ideaman may be on the right track. If this is a new gun it may need a thourgh cleaning, including the gas port. Since this is a semiauto, any fouling in the gas port may be causing slight differences in the way the action is operating and cause some variability in the accuracy. BTW, what size groups are you getting? The 74 series of Remingtonscan betough to get to shoot well. The one I had years ago was doing good to keep factory ammo much under 2" at 100 yards. A 1/4" of movement was nothing unusual but it didn't usually go in the same direction like you are describing.

Good luck with it. You may want to take the scope off and use the iron sights (if they are on there) at 50 yards to verify if the scope or mountsarea problem.