One place to start is -
http://www.landsofamerica.com/america/searchforlandbymap.cfm
You click on the state you are interested in and then the county within that state. They list several different types of properties but have a emphasis on vacant land. Sometimes you can find a realtor on that site that has other properties that are listed that aren't on the lands of america site.
As far as an average guy being able to afford to buy land, it comes down to priorities. Instead of the $6,000 ATV, that's a land payment. Instead of a new pickup, keep driving the old junker and put that $500/mo toward a land payment. Most of the Federal Land Bank systems only require a 15% down payment but that is still a chunk of change and the only way to get there is by saving.
I would love to have 500 - 600 acres, but I couldn't afford that so I went ahead and put the down payment on 160 acres and at least have my foot in the door. After I get this paid off maybe I can trade up to 320 acres by selling the 160 acres I have to use for the down payment. Maybe my neighbor sells his land and I get first crack at it. Unless you start saving, you won't ever be able to afford to buy land.