You set your goals and stick with them...Then you can always be proud of what you've done. But killing does is a responsible thing to do as well, and whacking one or 2 of those will gain you VALUABLE experience.
Doe hunting is an excelent way to gain experience.
Common theme, here.
You don't really know what you can do until you've done it. Gain the experience of laying one on the ground. I did the same things you're doing.....holding out....and I'm glad I did to an extent. But I still believe .....if I'd laid a doe down my first season (and I had the chances).....I'd have been better prepared when I had a deer in front of me I did want.
It can't hurt you to lay one down. As others have said....its valuable experience gained.