Exactly, practice is the only way you will become proficient. When you reach the point where you can shoot a 3" (or better) group at 40 or 50 yds), it will make those 20 yard shots seem sooooo much easier.
I shoot all year long. Even if its only in my basement shooting 15 yds and shooting only 10 or 15 arrows. This keeps the muscles toned and technique down.