goyard - I would also go with the 22LR in a manual action (lever, bolt, pump or single shot) and not a semi auto. The ammo is cheap and most rifles are accurate.
As for what 'deer' rifle to buy. That depends on your son's size. If he is on the small side you may want to consider a youth model for any of several manufacturers like remington, savage etc. And as for cartridge nothing less than a .243.
Again - you have many choices depending on his ability to comfortably handle recoil. Some good beginning choices would be .243, .260 rem, 6.8 Rem, 7mm-08, and the like. These won't pound that young shoulder too hard.
Congrats for having a son interested in the outdoors and shooting.