I don't think that the deer worry about fox. I regularly see fox when I hunt. I've seen them together, I've seen them in the same areas at the same time. They seem to ignore each other. At least in my experience. I enjoy seeing fox. They are fun to watch when they stay around long enough.
-Jake