It would upset the equilibrium we enjoy now, but dont assume a new equilibrium would not eventually be found. Those that lose in the transition will complain, those that gain in the transition will cheer.
I subscribe to the conservation of mass and energy approach to things, nothing dissapears, it either accumulates or changes form. Your monies you now spend for hunting will be spent elsewhere, or they will accumulate to be passed on to your heirs (minus taxes

)
Your time you spend hunting and thinking about hunting will be applied to your new hobby.
I am curious about the deer population however, because MO has seen a nearly 10 fold increase in hunting kills in the last 30 years, so hunting and the current plicies have not really kept the deer population down.