I do the whole layering act now, but just get sick of having to take it all back off around mid day.

I may just have to bite the bullet and buy a really nice wool suit... From my understanding, under armor plus the wool would be about all I would need down here.