Unless you're right on the edge of a flinching problem with 150's, the 180's won't cause a flinch where one didn't previously exist. That's a one in a million case.
I've always favored 165/168's in .308win, up to the 175 VLD and 178 Amax. 180's are fine, but I want a little more room in the case, so I generally run 30-06 or 300wm under a 180+.
You'll be fine with 180's, but as has been mentioned, you'll have to match their load, else you might as well start over from scratch anyway.