I would try it again, you can't really hurt anything if you already messed it up right? A tip I have been told works pretty well is dropping your backpack to the ground before you climb down. The deer hear the noise and it startles them, but doesn't scare them like seeing movement or smelling you does.
Either way good luck.