Determine what load you are going to be using for deer and turkey. Sight in with each making small reference marks on the scope dials to see where each shoots. That way when you switch from your turkey ammo to deer ammo all you have to do is adjust the scope to the spot previously marked and then fire a shot to check accuracy.
You should have no problems either way with a smaller dot.
As a side note you should be able to take a turkey at 35-45 yards with the correct loads if that is what you wish to do.