I have a place just like you are describing. The horses don't seem to bother the deer much as long as the people are quiet. If the horses are part of the regular landscape and the deer don't associate them with humans it shouldn't cause a big problem for you.