My father taught me to hunt the whole day. An hour or two spent in camp or at home or at the diner is an hour or 2 that you're not hunting. I've killed deer during every part of the day from right at sunrise to right at sunset and all times in between. You may need to vary it up some due to wind or conditions.
I was never a sit in a stand kind of hunter anyways. I've done it but I prefer slipping through the woods and stalking up my deer. Hunting all day works well with this method.