Yep it can get very frustrating sometimes, especially in areas that go downhill, or even worse, when you lose areas to hunt in.
The rut has saved many frustrating years though. As long as you have scrapes and rubs, those are areas the bucks are going to patrol and walk by, at any time of the day looking for a hot doe! It's all about putting in the hours! Thank god for the rut baby!