Buy a quality single stage press - new or used. Something like a RCBS Rockchucker will serve well through a lifetime of reloading (and often beyond). A good reloading manual, scales and calipers of almost any brand, and a few odds and ends will give you a start. Buy dies and shell holders for the cartridges you load, but you are probably better off avoiding the cheapest ones on the market.

I sometimes see good used presses for sale. Good ones are nearly indestructible.