Your bow will be fine, but as you add layers of clothes your form and string clearance may change. Practice with what you will have on while hunting.

The big problem I've had is the lube in the release freezing it shut. The only bow issue was ice freezing the cam in place from compacting snow in it; keep the snow dusted out of everything. I've had the sight get jammed up with snow too. The amount of clothing you have on will alter your shooting so be sure to practice in the actual clothes you'll wear out. Don't freeze anything off!