If he said you are welcome anytime, then go for it. I would try to get a hold of him anyway though to make sure that no one else is out there too. He may have given permission for someone else to hunt also. Non hunters don't usually understand what deer hunting is always about.