Ditch the sweats. They'll get wet from sweat and keep you cold. I hunt in the mountains of northern PA and our buck seasons can be anything from -5 in the morning during the cold years and up to the 40s in the warmest years. Our late seasons are just butt-arse cold, highs in the teens some days.
I go with a base layer of thin Under Armor, then some Cabela's Polar Fleece, then a wool sweater. I wear insulated bibs and a heavy jacket over that. I wear 2000 gram thinsulate boots and a warm hat or ski mask. I always bring a muff and stuff it with 4 hand-warmers.