I have never heard of too much rain affecting antler growth, but I can promise you that not enough can.

Where I am at that is a distinct possibility every few years. Things are not looking good here this year. It is a very bad thing when you have to fire up the irrigation wells just to get the crops out of the ground. About the only moisture we got this year came in the form of golfball to baseball size round spheres. Took our wheat flat to the ground.