Might give it a day to cool off but I think you can go back. I've been busted at spots before and seen deer again in the same area an hour later. Just make sure to keep your scent as minimal as possible in case the same deer return.