Deer don't decompose as fast as some other animals such as cattle or pigs. I imagine that deer was basically eaten by animals. You didn't say how big the antlers were, but fairly large antlers take about a year to get eaten by rats. I found a skull and nice sized rack of a buck shot by a neighbor about a year later and the skull was intact and the antlers had notches chewed in them by rats. Smaller antlers would be consumed faster, though.