Sounds like your being very responsible and only taking what you need. Those yearlings are very tender.
For me I try to take mature does or a nice buck if they come around. I'm trying to fill the freezer and I have many family members that I give my extra venison to. I try to get the bigger does just to get the extra meat.