Yes, deer can and will destroy your plot if the plot is relatively small in size in comparison to the number of deer in the area.