Will the deer shy away from my ground blind ( I dont want to leave it up all the time on public land) if I remove it and set it up every time?

some will... Have you considered building a blind outta deadfall? It doesnt take much to conceal yourself, and most likely no one would see it...

I definitly wouldn't leave it out on public land, so I'd go with Ed-- try to build something with whats already in the woods.

Some will and some won't it depends where and how you set up.
I use a Double Bull ground blind when I'm not in my treestand.
Sometimes it might be easyer to use dead fall but you want the comfort and protection from the rain and wind then the a ground blind is a good choice.
When I set my blind up I'll put it back in some thick brush and cover itup alittle with some branches and fresh limbs with leaves to make it look more natural.
Look at the photo below and you'll see what i'm talking about.

Thats my Double Bull ground blind when was spring turkey hunting this year.
Not only I had some turkey come in but I also had deer walk within bow range as well.