Brick wall? Unless you stood and drilled it with every arrow then that's not a problem. Something I haven't seen mentioned is the rigging (string/cable) on the bow. Assuming your bow was well tuned before and your arrows were shooting really well, what could have happened, and often does, is that the string and cable have crept. Usually the string creeps more and gets longer. The draw length gets longer and the peak weight goes up. This throws the bow out of it's original tune so now the arrows don't fly the same. The cure is to have the bow adjusted back to factory specs and retune it. Then take all the important measurements so you can check periodically to see if it changes again.

As for going to a lighter broadhead? It can gain you a couple fps and help the arrow flight if your arrows are borderline underspined. But you still have to pay attention to details such as aligning the broadheads with the shaft and mounting them on stright arrows in the first place. Once all this is done I would suggest a slightly softer backstop than a brick wall.