Deer definitely eat them. If you are talking about these:


I have always heard these are one of the last things they eat when other food sources are scarce. When I go out shed hunting I have a few places where I know there are these type trees and I always find more sign there than anywhere else in the entire area. I always find multiple smashed hedge apples and it sure looks like they are feeding on them.