According to all evidence, deer do not see color. Most camo is based upon human perception of color.
Yes, breaking up your pattern is helpful, but I'm sure you'll get dozens of stories of people who had a deer within 20' staring at him or her without camo (I have a couple, deer and elk). Most important is using the wind. After that breaking up your form is a bit helpful (camo). But it's not necessary. Good outdoor clothing that is comfortable is a better investment than cotton camouflage.