RE: Old .22 single shot bold action rifle
Congratulations son and welcome into the sacred circle of the smoke stick. Many of us have an old bolt action .22 handed down by fathers or grandfathers.
As far as rust removal there are a variety of things. WD 40 removes rust from bolds and will from your gun. many of the solvent based cleaners will remove some level of it. brake cleaner will. bet if you asked your grandfather he would have a solution for it.
swab the bore and look down it. get it nice and shiny clean. odds are there isn't a thing wrong with the gun. Again Congratulations.