I've heard two things when it comes to Goats-
1) IF you shoot them when they're running, they have a bad taste.
2) If you shoot one that's bedded down, not scared, and take care of the meat properly that it can be cooked just fine and has good flavor.

I've never personally had experience. That is from my dad and grandpa who used to hunt wyoming all the time. Good luck!
-Jake