It used to be that 20s were on the light side for deer. The newer 20 Ga slugs are perfect for deer.
From Federal's website, their 20 Ga slugs rated for 1900 fps retain more than 1000 ft lbs of energy at 150 yds. Sighted in about 2" high at 50, zero is 125 yds, and less than 3" low at 150 yds.
H&Rs are a heavy gun in general because of their thick-walled barrels. They use a 10 Ga barrel blank bored to 12 Ga, and the 20 Ga starts with a 12 Ga blank. This makes a stiff barrel and contributes to accuracy. But the gun is heavy. If the ballistics above meet your needs, then go with a 20 in the H&R. The 12 is only going to be a heavier gun.