Yep, it's just possible the bow might not be tillered properly and is fighting you. Not as likely as TP from what you describe, but it's something to try.

Here's the way I'd do it though. Put your sight pin on a spot and begin drawing the bow, straight back. The bow will likely try to pull the pin off the spot as you pull, either up or down. You make adjustments on the limb bolts based on which direction it goes. I'd suggest working in half turn increments at first. Then as you get closer you can make smaller adjustments.

If the pin pulls UP, tighten the bottom limb, loosen the top limb, or both.

If the pin pulls DOWN, tighten the top limb, loosen the bottom limb, or both.

Keep adjusting until the pin stays close to the spot as you draw, all the way to anchor. When you've got that done, you've got the tiller adjusted to match your unique grip style. The bow will hold a lot steadier for you at full draw. It will also draw smoother and very likely shoot faster and quieter than it did before.

Just so you won't be taken by surprize, this procedure WILL cause your nocking point to move. How much it moves depends entirely on how badly your bow is tillered right now. Be prepared to adjust it when you're done.