I agree with the advice to hunt the routes into and out of the area being damaged rather than hunt over "bait". And don't forget that deer are very nocturnal. I suspect that is what is happening in your case. If you have a trail-camera, try setting one along the trail into the damaged crop. Might answer a lot of questions ?