First of all, in my experience, 25-06's can be fairly finicky when it comes to ammo. While this can be true of certain rifles in dang near ANY caliber, it seems to exemplify itself in the 25-06 more often than not. And by finicky, I don't mean that it will shoot Federals great but doesn't like Remingtons...oh no...not quite that simple, more like it won't shoot any factory ammo much better than another, but certainly none good as it should (think around 2" or so at 100).
It sounds like you are judging the cartridge based on a single gun,which is by no means a fair judgement.A hunting partner has a tikka t-3 in 25-06 that shoots virtually every load that he has tested into less than an inch at 100 yards.A few loads constantly do 3/4" or better.I have never seen a 25-06 that was as bad as the one that you describe.