While many might tell you that you can improve on a 3" group at 100 yards, there are many out there that secretly are tickled to death when they shoot a 3" group. Scope or no scope.

Second, you're just getting used to the load and the bullet. Maybe even the rifle. It will get better with practice. I too would suggest some loose powder. Start with 90 grains of loose and work from there. Also do not be afraid to try other projectiles.

I never shot the Remington you speak of but I have sure heard a lot of good things about them. So keep up the swabbing, practice, try some loose powder, and don't be surprised when the groups start to shrink.