Line dry it rather than using a dryer. Which is what you ought to be doing anyway. However, don't leave it out in a hard sunlight for too long. I find that the shade on a nice breezy day is just the ticket.

In the winter, I usually (if possible) hang dry mine by a wood fire..... your clothes end up with a smell of wood smoke, which is a 100% natural smell. Does it work.... heck I don't know.... but I like the smell of it....

Moreover, there are scent free detergents (that aren't designed for hunting and don't carry a designer hunting price tag either) that contain NO UV BRIGHTENERS, but also have color lock or some other type of BS... I like Purex (unscented of course)....

I actually want my clothes to fade a little bit.... I find that (especially for tree stand hunting) that a slight fade to grey helps them blend in MUCH MUCH better..... A slightly faded realtree HD green..... I can't see it.