Killing the tree isn't too big of a problem unless you aren't removing the spikes each year. The poisoning of the tree from the leaching of the metal takes around 4-5 years to begin the process, and it will kill the tree shortly thereafter, so I never leave them in for more than the month of archery season.