If your a trophy (antler) hunter, ya gotta judge him accurately and if deemed a bit young, let him go til next year. Of course, as you posted, there are many, many more variables which you have absolutely no control over, so once you decide you live with it, possibly forever. The deer may go off and wind up dead many years later from no teeth and starvation (old age) with you never seeing him again. So ya never know if it's the right decision UNLESS you see him again., positively ID him and take him with more antler.

If your not a trophy (antler) hunter - he'll probobly taste just about the same next year, but too old and they can get a bit toughter.