You haven't wasted your money on that goose down set up , you simply haven't encountered it's ideal conditions . Get a synthetic and roll something like a simple tube tent up in it for added shelter when you decide to set up camp for the night . Perhaps what you need is more along the lines of this:
Covered hammock
or this:
A different one
Combine either one with a properly rated synth bag and you're set .