I can only assume the idea is based on being more camouflaged? Given that your wearing an orange hat and your wife is wearing orange as well this doesn't seem to matter. I'm sure most will agree that the orange isn't usually the issue, it's scent. I've had deer walk right by me in my big orange suit and didn't spook until they got my scent.

I'd err on the safety side myself. private land but that will be small consilation if a trespasser shoots you trying to poach a deer.