If ya can't postively ID the target, don't shoot! From your discription, it sounds like you guys have some doubts about what you shot!!
BUT to your question, coyotes can make a lot of differant sounds, many of which some people can't tell the difference betweem domestic dog and coyote. So, yes it's possible that a wounded coyote can make sounds that sound like a dog!