My father and his buddies used to hunt years ago in jeans and plain clothes and never seemed to have trouble filling the freezer.
My dad killed his nicest buck ever.......a real nice 8 pointer while weraing jeans, a flannel shirt and shooting a double barrell shotgun
I think camo is just fine to wear but I also believe you could see just as many deer without it.
Like many have said already........movements and sounds will get you busted 99% of the time.........not what color pattern is on your shirt.