Assuming that the bullet exits the animal, then the ONLY way a bullet can make a smaller exit hole than entrance hole is if the bullet totally fragments and only a small piece exits the animal. Even with limited fragmentation the mushroomed bullet will be larger on exit than entry. In all other cases the exit hole will be larger than entrance hole.

If you are seeing exist holes that are smaller than entrance holes on a regular basis you should probably switch to a tougher bullet.