Don't beat yourself up! There's nothing wrong with taking a young deer. Especially once you've tasted it. That link posted is a great idea. Also, if it does bother you, just look for a bigger one next time and learn what to look for. I know guys that take the smallest doe they see once antlerless season rolls around. As long as it's legal, no worries.
-Jake