There is only one reason to hold off for a 4.5 yr old buck. That reason is that the age will produce and exceptional set of antlers (kind of why nobody ever talks about shooting a doe of X yrs, just weight). If you are able to hold out for a mature deer and he presents himself with damaged goods you should be able to hold out a little longer.

I have shot enough bucks with broken tines and beams in my past that I would like to say I would pass, but I obviously haven't done that yet.