You are perfectly fine. The deer won't leave at all. They will smell you maybe, but they won't stop using the area. They will come back faster than you think.