If you have more his size then let him walk! If you shoot based on what your neighbors may do, then your progress will be nil! It's always cool to be able to say you could've shot that deer even if your neighbors do shoot him.