If I use a pop up blind that I throw up that morning/afternoon, at least half the deer see the blind and give it a good stare down. It doesn't seem to spook them unless they see me moving, but they definitely have their guard up.
On the other hand, last year on opening morning I hunted out of a permanent blind that had been there for at least three years. I saw multiple deer and none of them even thought twice about the blind. I even had a 9-pointer walk close enough that I could have reached out and poked him with my gun (when I saw him taking the trail towards me, I ducked down inside the blind because it was a see-through material.) No joke, the deer was within two yards of me and had no clue I was there. However, no deer had been shot out of the blind for a few years, and it had only been hunted a handful of times. Maybe that's why they didn't think twice about it.