I don't think the color of the rack has much to do with the age of the deer. I just got a buck that had a pretty white rack, but the age of the deer was about 4 1/2 years old. At least thats what my taxedermist said. He judged the age of the deer by it's teeth.
Check this website out, it will help show how the deers teeth change as their age progresses.

http://www.whitetails.com/deer_info/age_determination.cfm