The best thing I've found was to gently rub the affected area with a dry cloth or paper towels to remove as much of the offending oil as possible, then slather on some 1% hydrocortisone cream like Cortisone 10 ointment. The hydro keeps the itch to a bare minimum and helps to prevent the inflammation. If you're already blistered it drys up the blisters almost overnight.