I have hunted 30+ feet off the ground, 10 feet off the ground, and hunted on the ground itself. Every setup is different. You definitely don't have to get over 20 feet to shoot a deer!

Let me preface the fact I hate ladder stands. I think they stick out like a sore thumb unless they are concealed properly. I have four stands out this year and only one of them is a ladder because it is my brother inlaws. It was a pain in the ass to hang and even harder to conceal. Height for me is regulated by necessity. However high I need to go to have the correct cover is where I will be.