The front site is normally on a dovetail wedge. A brass pin and hammer will drive them out. It's been a long time and I might be wrong, but I believe they drive out left to right.

The wedge barrel lug you need to examine close. Some of them have a small spot weld on them. If not, they too are in a dovetail of sorts. I normally do not mess with them. I let the factory play with them if there is a problem.