You made a good call,Usually the young ones should be old enough by now to fend for themselves.
I usually waite and see if there are no young ones by doe I harvest it.
I have found that they have milk in their mammery glands at times as well.
Don't feel bad about your decision,Like I already said,The fawns from this year are big enough to fend for themselves.