I believe in going higher. I get my climber at least 20 feet high, usually around 25 and sometimes 28. I have markers on my hoist rope so I know how high I am. I do have 2 ladder stands that are 15 feet and one that is 20 feet. While sitting in the 15 footers, I usually have deer look at me. As far as being silhouetted, pick a tree where that doesn't happen. I have used trees were I didn't have as much cover as I would have liked but the deer don't look that high. I feel that in my area, there is so much hunting pressure that the deer normally look up 12-15 feet high.
As far as getting the heebie jeebies that high, you get used to it. Just make sure your harness is tight. I have mine set so that while sitting, if I lean forward, it pulls on my harness.

Here is a shameless plug. Since I have been using my ASAT 3D leafy suit, I have not been busted. This thing is great. If you have to stay low, I recommend getting one.