Hi BillyMack,
Ask one of your game wardens or contact one of your state biologists by phone or e-mail. They should be able to give you an estimate of the buck to doe ratio in your area. Your dead on about them not moving around when there are plenty of does around. Cuts down on the other parts of the rut. Less scrapes, rubs, and "fights". If its as bad as you think, You could drop every one you see and not make a dent without some help around you. Don't know how big an area your talking about either.
Good Luck on that 1st buck.