Nope. That's normal.

This is an example of someone that doesn't fully understand their weapon, but is trying to brainstorm a way to tune out a "problem" that is normal function. The top of the lower is the positive stop for any charging handle.

If you file down the top of your lower extension mortise, or file down the bottom of your charging handle, you'll just get more slop, and it'll make the same marks again.

Remove your upper from the lower, unlock the charging handle, pull it about 2" rearward, then you'll see how the charging handle can pivot downward with no resistance. That's because the rear of the charging handle is not supported by any trackway - it is designed to ride on the extension bridge.

The "knobs" on the charging handle near the front lock into the track in the upper, but there is no such alignment lug at the rear - hence the belly of the charging handle HAS TO RIDE ON THE LOWER EXTENSION BRIDGE!!!