Check the loose dirt for sign, I can almost bet they are still in the area. Deer are curious by nature and if they do not feel threatened they will not be going anywhere.