Are you target shooting or hunting?
A PRB runs out of energy quickly. I'd keep hunting shots under 100 yards with a .490/.495 ball.
If the sights are not adjustable I'd pick a load and check it carefully for accuracy before I got in a hurry to take a file to the front sight. That's the way to raise POI, but once you take it off, you can't put it back if you change your load.