Actually your first step will be completing a hunter safety course, and then you can get your hunting license for which ever game you so choose to pursue.
And a rifle may or may not be the best choice, depending on which game and how you choose to pursue them. A 20 or 12 gauge shotgun may serve you better than a 22 rimfire rifle.
You may learn more about your areas game and the best methods of hunting them from the hunter safety course. SO don't rush on purchasing any gun, until you have a better idea of what game is in your area to hunt, and what game you decide to hunt.
Contact your local wildlife department to find out more on when and where the hunter safety courses are available. Good luck!