I do have a .204. It is a Ruger M77 MkII, Target. I have a farmboy buddy who shoots varmint near daily. He had been shooting the 22-250 for many years, but swore to me that the .204 was a better shooter. Coming from him carried a lot of weight for me. I had read that they were pretty comparable, and bucked the wind pretty well. Very hard to believe! I don't shoot in a tremendous amount of wind anyway, but the .204 did just as well as the .22-250 that my father and Uncle shoot (at prairie dogs). Of course, we don't do the 400 + yards. 100-400 for us.
I have shot both, and am not convinced that the .204 does as well as advertised. I still believe that the 22-250 if superior in many ways. But I do very much enjoy the .204. I have found no downside yet. But, the most favorable thing about the .204 over the 22-250 is its appetite for gun powder. I reload both, and the .204 is cheaper to load.
I am dying to try the 220 swift. I keep hearing the good, and drool!
I think that you will love the light recoil of the .204 as well!
Enjoy!
Victor