I have had this happen many times. Maybe since there's something there to eat, the deer simply wait for the yote to leave then go on in to get their "goodies."

Blessings.....Pastorjim