RE: What to wear?
I can't speak for bow hunting, but clearly camo is not necessary for ALL rifle hunting. Where I hunt I position myself in a ground blind among trees overlooking a trail. I locate where I'm in shadows, there is clutter behind me breaking up my outline, there is clutter in front of me screening my legs and lower torso, I wear a face net I buy cheap from WalMart to keep my face from "flashing" as I turn my head, and when I move I move slow. I hunt with the wind in my face. In these circumstances how much difference would wearing camo make? Last year I hunted in a red-plaid flannel shirt and green dockers and took two deer -- a doe and a buck, my bag limit in Oklahoma. The previous three years, while wearing camoflage, I only took 1 deer each season. Not statistically relevant, I know.
As you point out, how were hunters able to take deer years ago before the various camo patterns emerged?
Use your head. Other factors of hunting dominate over camoflage.