I've seen average deer rub on big pine trees, which appears to be the type of tree you have. Maybe it is smething about the tree that attracts smaller deer to the bigger tree?

Nice buck though.