Don't worry about it. Deer aren't afraid of horses, or they'll get used to them quick.

The deer are always in the pasture with my horses. My stand is in a section of woods that has pastures on one side - I see the deer mill out to where the horses are quite often. They won't let a horse get near it, maybe 50 yards or so, but they just move somewhere else in the pasture and go back to grazing.