About silver skin- I have found that removing as much as you can from solid meat helps when it comes time to eat it but that it does not affect the flavor of the meat so we do not worry about it in the trim pile that gets ground.
Just a word of caution here, if you cut your own deer up and take it to a locker for grinding some of them will want the silver stuff and all the fat removed before recieving it, I know at ours they charge a pretty good fee if they have to clean up a bunch of meat someone else cut up.
They claim the silvery stuff clogs the grinders up, Ive noticed it on my grinder as well but its just a little one.