Hunting is supported mostly by the weekend warrior. Not everyone is the die-hard maniac. These are the ones that will get fed up with the high cost of leasing land and over crowded public land and quit. I also believe more outfitters will start controlling more and more land. If you want to hunt you hunt through them.
More people are moving to cities and the rural lifestyle is getting phased out. This means that more of them will not care about hunting and the anti hunting folks will become more powerful and will start changing policy.
The farmers once needed the hunter to help keep deer under control but now they see the hunter as an form of income. For now they are making extra money but once the hunters stop controlling the deer numbers they will be hurting from all the crop damage. They in essence are shooting themselves in the foot.