Speed gain would be minimal. The worst thing that can happen is you lose valuable FOC which can not only affect your penetration but also can affect arrow flight and group size. If you have the spine to carry the 125's I'd stay with them.
I'm shooting the New 125 grain Grizz trick this year.
Dan