That is the eternal question. Not one that anyone wants to answer. For me its not when they are in pain, or happy or not, or can't stand up. Its when they lose their own dignity and I can see that they are "embarrassed", for lack of a better term, for not being able to do the things we always did. I am sure I will get shelled for saying that, but go ahead, I can take it. You spend enough time with a dog, you can see these things. It becomes selfish for me not wanting to go through of pain of death.
I can't shoot my own dogs. I take them to a vet and get the shot. But I bury them myself. They are buried wrapped in my hunting coat, with two shells in the pocket, and a bird in their mouth. Just the way that makes me feel good about the whole thing.
I tend to have to buy more coats than most.

But since you asked. And I don't know you sir, but if your dog has to wear a harness to get walked, it has lost his dignity, and that tail wag, that is the embarrassment I am talking about. Let him go.