I reckon it should be done when the deer is hanging. It would be much easier than after it's cut up in chunks.
You cant take the silver skin off what you can't see. (ie: the loins.) Iremove it as I'm butchering. and if you miss a little you can always touch it up before you cook it. All meat for grinding is thoroughly cleaned of fat, silver skin , tendons and sinew.