Yes its worth it. One thing to remember, when you pass up young bucks and go for older ones, you have to change the way you hunt. If you continue to huntwith an"any" buckmentality, you won't have as much success at the big ones.
But again, every little bit helps with passing on younger bucks to let them get old.