Well if you can anchor your thumb on your neck that is your first problem. Either your draw length is too long or your release is too long, basically having the same effect. Besides throwing your form off anchoring your thumb to your neck is a poor anchor and not very consistent.
50 yards is getting out there, don't expect to shoot well at that distance unless you have really consistent form and good release technique.
My first question is are you using a peep sight? If you are I wouldn't suggest a kisser button. If you are not using a peep sight then I would try one, but I would suggest the Timberline NO PEEP instead. Much better product in my opinion. It does cost quite a bit more than a kisser button though

.
If you are using a peep sight, the location of it and the size of it will play a role in how accurate you can shoot with it at longer distances.
My opinions anyway.
Paul