I typically write it as "brake" because a brake is an object which slows velocity, and the muzzle brake slows the recoil velocity of the rifle by redirecting (and slowing) the propellant gases. A "brake" is a primarily a noun, and is only a verb when describing the action of slowing down, "I braked when I saw the deer run in front of my car."

"Break," alternatively, is typically a verb, as in "to break something," as in "break action shotgun," which breaks, or hinges. When used as a noun, "break" typically describes the location of a verb form break having happened - as in "how did you break your arm?" and, "where is the break in your arm?"

So to me, it's a "brake," which slows the recoil velocity of the rifle, and since it is attached to the muzzle, it follows to be called a muzzle brake.