Sounds like you didnt spoke him. He just didnt want to be there at that time. He will be back. If you see him all depends on if you happen to be there when he is. With all that said, I guess it also depends on how you left the place. I assume you didnt stink the place up and snuck out like you snuck in.

Just dont over hunt the area. Only hunt when the wind is right.