There answer is yes with a disclaimer. I would take a shot at a big buck that I wouldn' t take at a doe. However, I will not even shoot at a doe unless the shot is perfect.
I agree with Silent.
If I were to shoot a doe or a dink buck - the shot would have to be a dead slam dunk. I know myself - and if the shot was bad - that could equal to 3 sick days from work and precious time tracking and searching. I would put in the days of tracking for any deer - but would be pissed that I had to on a doe or small buck.
All my shooting limits are set up for good Bucks (God willing, its been a slow opening week

)
They get no bigger for bigger bucks but they do (or would) get a little easier for smaller deer.