A heavier point will reduce the effective spine of your arrows--basically it will make them shoot like a weaker spined arrow. Adding weight will have an effect on your speed, but it shouldn't be much. Switching to wood arrows won't matter--you still have to have the proper arrow stiffness (spine) to get proper arrow flight.

Chad