I think I'd raise it up if it's jumping that bad. If you have to go to 6 1/2", I don't think it'll hurt, but naturally the lower you can go and still get acceptable shooting the better.

How does the tiller look? Sometimes a bow will thump the hand a little too hard if the tiller isn't quite right. You might need to to a little additional sanding to one limb or the other.