As far as I know, and I own three IR cameras, it doesn't seem to actually spook them but they do stand and stare at them. I get the same reaction to my white-flash cameras so I'm not really sure there is an advantage either way. Now there is supposed to be something called black flash on some of the newer cameras coming out that is supposed to be completely invisible but I haven't seen or read any feedback on those systems yet. One possibility is they are hearing the sound of the IR filter. Some of my IR cameras make a slight sound when taking a picture. But then again, they also stand and stare at the IR camera that makes no sound when it's taking a pic. I'm sure that you're as confused about this as I am now.
Blessings.....Pastorjim