You should be thinking safety, not regs. Me? I can't imagine why you wouldn't wear as much orange as the shooter. The only reason I can think of is that you don't want to be visible to the deer. If this is the case, buy yourself a cheap blaze camo vest and hat.* They are pretty much as effective as regular camo for deer, imho. I am of the school that believes that deer see blaze as a shade of grey or brown and so that a simple break-up pattern on the blaze is an effective pattern. Regardless, your movements and scent are going to be 100X more important than the color of your vest.

*I have no idea if blaze camo is legal where you're hunting or if that particular reg would apply to a non-hunter anyway.