I will assume outside rust and a gun that is not a collectors item. I would try a cloth with oil and rub. If that does not work, very Very fine steel wool soaked with oil, and used as lightly as possible might do the trick. My idea would be to try not to rub the rusted area too hard, just enough to remove the loose rust and make smooth , but do not polish to bare metal.

It is very hard to make suggestions without seeing the gun, and if it is important to you, take it to someone with experience the price will be worth it.

It goes without saying, but be sure this gun chamber and barrel is inspected well before being fired.