RE: Correct spine
To answer your question- yes- the spine of your arrows can make a big difference.
If you're shooting your aluminums with accurate results then your shooting shouldn't be the issue. The issue sounds like a spine issue to me.
Shoot a group pf 3-5 arrows each back to back and see what happens. If your shots grow with the carbons then it would be safe to say they are not spined the same as your aluminums. Assuming the fletching type & orientation is the same (incase one is hitting & the other is not, or feathers vs. vanes).
Your solution could be as simple as adjusting your fieldpoint weight up or down by 25 grains on the carbon arrows.