I'm thinking along the same line as Rack. Although it's nice to have the sun on your back I'd be more concerned about the prevailing winds. If the wind is in your favor where you're at I stay put and "fix up" your tree. If the other spot is as in favor of the wind as your current position or better, definately give it a go.

Do you have a climber or a hang on style stand?