Adding more fletches while it does add more drag surface also adds more weight to the back of the arrow.
More weight obviously means less speed and being in the back it can also have some effect on FOC.

That being said, using 3 fletches and putting some offset or helical on them will get you what you need with respect to spin without the need to add the weight of another fletch.

The other consideration is that arrow trajectory and accuracy can be adversely effected by too much drag. As the arrow slows, the fletches do less and less work, causing the arrow to potentially stray off course. Your mission, should you decide to accept it, is to find that sweet spot, that stabilizes your arrow effectively while maintaining enough energy downrange.

In general 3 fletches will get you where you need. If you want more stabilization, use more offset or for the ultimate, use helical.