It probably depends. If you are talking about an isolated area where vehicles are rare, then the vehicle, plus any human scent that you leave behind when exiting might be a problem. If you are talking about a public land parking lot that is frequently used by hunters, fishermen, shroom pickers, hikers, etc., then the deer are probably used to it an don't pay any attention.

How hard would it be to accommodate the other hunter's request? Even if he's wrong, he believes the truck is the problem, so is parking there worth the bad-blood?