Could be a rub. I have saw rubs on telephone poles and those are large in diameter also. Made by a mature deer for sure. If you say you have big ones around it surely could be a rub. Beams come around the tree and hit the backside too. Hopefully he you will see if it indeed is a buck!