There is a lot of truth to what you say. I've read at different times about riflemen in each of America's wars here and overseas that some of the best riflemen were those who'd grown up around firearms, hunting on farms, ranchs, small towns.
I saw that in my time in the Army that those who knew firearms and had hunted squirrels, deer, etc. from a young age were general those shooting in the top of the companies. Certainly there's more training for situations Soldiers,Marines and Special Forceswill encounter that prepare for conflicts with those shooting back that's different than hunting animals, too.