As was mentioned, this goes to show you how fast things can happen. This is why I don't read, play with my phone, etc while on the stand. Chances are you'll miss an opportunity if you're farting around and not paying attention (which didn't happen in your case, BTW).
If you can change trees or stand locations slightly, that might help. He's young and will still come through that area, but he might check out that tree the next time he comes through (I've had deer do that before). Next time, one thing that might help is to take along a doe bleat can. I've been able to reassure deer that have gotten a little spooked by giving out a bleat or 2. Got one this year that way (he spooked, ran off 20 yards, and a bleat calmed him down and brought him back in).
I also agree with nodog that the wool coat shouldn't have given you away. More likely he either smelled you or caught some motion. I swear that deer can see your eyes moving, and for that reason I refuse to make eye contact with them when they're close. Think about it...if you're in a crowd, you can pick someone out of 100 people if they're making eye contact. I have no doubt deer are the same way.