I have saw deer walk by me while sitting on my activebear bait. Black bears only occasionally take deer, I would not worry about it. Are you are worried the bear will scare the deer away? They won't unless you have serious black bear problem in that particular area. And here it is illegal to have any firearm in possesion while bowhunting. Just jab him in the eyes if he charges.

Most likely will be a bluff charge anyway. Just back away slowly.