In my opinion, 11:00 to 12:00 is one of the best times to be in the woods turkey hunting. I'm like you though, I love to hear him gobbling on roost before flydown. Obviously this is when they will gobble the most. The gobblers very well may be looking to find a lonely hen after the rest have been bred or go to nest. This is what makes that 11 to 12 period great sometimes. I've killed two nice birds this year, one at 11:00 and the other at 12:26. You may miss most of the gobbling, but you will be in the woods at a good time.