If you don't want to use wraps, you don't need to. Perhaps you're just not using quite enough glue, perhaps you got a bad batch of vanes, perhaps you don't have your jig set correctly. If it's not set correctly, the entire base of the vane won't sit on the arrow and they won't stick. Check that before you do anything else.

I clean my shafts with 91 % alcohol before placing a 3" wrap around the shaft. Wrap the shaft. Then I apply LOC Tite brush on Super glue to the 2" blazer and stick to the shaft. Never had one come off yet. This is just the way I was taught, I never could get the fletching glue to stick properly. Wraps help alot and are a good visual on shot placement.