If the deer are not used to seeing the ground blind then the risk is they will be wary of it when they do see it. If they are used to it being there then scenarios such as rain hitting the blind won't be an issue.
That's the advantage of setting it up in your hunting area well before you plan to hunt out it in that location. If you don't have that luxury then all is not lost. It's just not ideal.