I could keep a 2 inch group with his and a 4 inch group with mine. He was using less powder and of a different kind.
Well, why don't you try duplicating your buddy's load for starters? I am assuming you tried his loads in your rifle. Each rifle is unique, and while there are certain loads that seem to perform well in a lot of different rifles, there is probably a different load with different powder and or bullet or even primer that may provide even better accuracy. Some rifles in 30-06 seem to prefer faster powders like IMR-or H-4895 or IMR-4064, particularly with lighter bullets of 150 gr or under. My old 30-06 shot well with the 150 gr bullets with IMR-4895, Reloder 15, and Accurate 2700. It also shot well with H-4831 and Reloder 22 with the 165 gr and 180 gr bullets. My rifle prefered Nosler Sollid Base and Ballistic Tip bullets, but also shot reasonably well with Speers and Hornadys. Good Luck.