The ears my friend...

A tell tale sign of a button buck is abnormally large ears for the size of his head. He will also show much more black on the back of his ears.

To me this is the easiest way to tell. I can spot a button without binos from many yards away just by looking at the ear size.