Just hunt them. You do need to worry about them being there early before dawn and after dark. You will push them out of there. You could always wait until it gets light out to get into you stand to make sure your not cgasing them away. Otherwise you should have no problem with you scent if it's near the house where people frequent anyway.