I'd prefer the .22mag over the 17hmr. But I usually pick heavier bullets over velocity. I've only used a shotgun on yotes once, and was pleasantly suprised at how well good ole #4 buck works.

If you plan to do more hunting at around the 75-100yard mark I'd definately would choose the 22mag. The shotgun isn't going to do any good at those ranges and the light bullets of the 17 begin to fall behind the 22 at longer ranges.