I believe deer do have a memory. I also believe the bigger they are the better it is.
Seriously, if they didn't, why would they spook when they smell a human? Why would they not spook as easily when they inhabit an area close to houses? How do they know that a penned dog is not a threat? And so on and so forth. Deer will pattern hunters as well. How many times have you been out riding the day after season goes out only to see a really good buck in the middle of a field. This means they have a subscription to the local DNR rule book.