To me it doesn't seem like you spooked him enough to know that you were something bad in the tree. I had this happen this year with 4 very mature doe's. I seen these same 4 doe's back to back nights, let me add that they were the biggest doe's i've ever seen. They checked me out in the tree also, my wind was kinda catching them both nights but they couldn't put it quite together. I didn't move an inch but i know the second night out hunting i could of zipped them again! An old doe can be just as smart as some of these nice bucks were seeing. Good luck to ya!