I agree it's the movement rather than the color that disturbs them. If I hear another hunter in the wood I'll put on a orange hat and I keep a vest in my pack just in case, but at the very least I clip an orange baseball cap to the back of my pack. I don't care if I miss a squirrel if it means I don't catch a stay shot by accident.