It is very possible to cull for spikes and have antler size decrease. Drought and other conditions will do that. That doesn't make culling wrong, and in fact it doesn't prove that it is a poor management decision.
It sure doesn't prove it'sa GOOD decision.
With a spike, the genetic potential is just not there. That buck will never have the potential of a deer that has 5 or more points in the first year. That means they are an unproductive use of the resource. They are consuming resources that could be going to that 1.5 with 5 or more points, which has much greater odds of becoming a real trophy. If you don't get rid of the animals that aren't efficiently using your resources, then you are wasting time and money with any other management practices.
Is it 1980 again? It's been proven over, and over, and OVER that culling deer based upon antler traits is worthless. It's nothing more than a advertising gimmick used by deer "ranch" owners on television shows. I guess some people still buy it.