The reason why he is going somewhere else is he's got a girlfriend going that way.
When you answer him a soon as you can you might get him to think you are the hen of the day, and come to you instead,besides sometimes the hen is roasted somewhere else. If he is gobbling going away, just keep on answering. He'll take note of your position and come back later when the hen he is with is laying an egg. Don't move from your spot but call every now and then. There could be another one silently coming to check you out.