They burn too slow, they won't build enough pressure, you could get squibs. There are some powders that work in a rifle, like 'lil gun which you can use in .44 mag and in .22 hornet. Mostly the burn rates needed are too different.

For each case volume, there is a burn rate that will produce max pressure and a full case. Faster burning powders produce max pressure, with less powder. This produces less gas to push the bullet and therefore lower velocity.

Slower powders fill the case, but won't reach max pressure, so velocity is lower.

Check out a max charge of H4895, imr4350, and H4831 in .30-06 180 grain bullets (Hodgdon Website)

H4895 is too fast, you can only put in about 47 grns to reach max pressure.
imr4350 is about perfect, a compressed charge is close to max pressure.
H4831 is too slow, a compressed charge doesn't reach max pressure, velocity is lower.

Using rifle powder in a handgun cartridge is like using H4831 in a .30-06, only about 10 times worse. I suspect you'd have problems even getting it ignited properly.