You did the right thing and you don't have to feel bad about it. Following the law is not anything to feel bad about. It just was not in the cards.
Many years back because of the doe population out of control we had to shoot does. And the first late afternoon while hunting from my blind, I had three nice bucks come through and stand there. I enjoyed seeing them. But always wished I could have shot the second one that came through. But I did get lots of does that year.