Actually its not really an "off-the-wall" question.. You never know until you ask.....

As others have stated, your cat, or cat hairs on you will not bother the deer. I have 5 outside cats and atleast 10-12 deer in either my yard every morning and evening, the cats do not bother them. I do have to lmao on occaision when one of the cats gets a tad to curious and one of the deer will run it off the hill full tilt boogie!

Deer also ignore my Wolf/hybrids that are on cable runs outside and there is even a little button buck that plays with our Argentine Dogo pup in the evenings. ( She is 9 mnths old and weighs 87 lbs. the button buck is from last Spring, now.... that is a hoot to watch. ) Guess I need to get a new video camera battery.

Be more worried about covering your own scent.