Only potential problem I see with that is that you will have a fairly small pellet count using #4s in a 1 5/8 ounce load. I suggest patterning one at 40 yards to see if you have holes in the pattern. Remember also that the Hevi Shot isn't uniform in size. Someone on the NWTF board disected a Nitro 4X5X7 and found every shot size from size T to #9 in that thing.

I much prefer #6s with Hevi Shot simply because of the higher pellet count.