Turkeys don't really care so much about color, but they see movement VERY well, and if you wear blaze orange and move, they'll see it better than they would if you movedif you wore camo, since blaze orange doesn't match or blend in at all.
So, IMO, you shouldn't wear hunter orange, but using a flag might actually spook them if they see it flapping in the wind. Just go out therein your camo, and pray........