Steve,
I don't see anything wrong with that thought process. I've held off of smaller bucks for years, but I'll tell you I still do get VERY tempted!!! If you're happy with the animal you shoot, it's none of mine or anyone else's business unless it's illegal or against a landowner/club's wishes.

Good Hunting,
Doug