I wash my clothes in scent free detergant too, but i try not to use the dryer if possible. I like to let mine dry out on the clothesline. Sometimes thats not possible in the wintertime. So if i do use the dryer, i make sure to use some of the scent free hunting type dryer sheets and then let them air out afterwards if possible.
I just think there is too much perfume type smell associated with the dryer.