Check these hand blankets out. I have used the same one for 20 years or more.
http://www.icebreakerinc.com/php/hanBlanket.php
You can get it with or without the compartment for a hand warmer, but I've never needed one. I use that and just have some light gloves to use when stalking or climbing into a ladder stand or climbing stand.
You can find them at a Gander Mountain too.