Yes, it is a concern and melting lead should only be done in well ventilated areas. The vaporization point of lead is high enough that most consumer casting pots aren't going to get you there - but ya never know. Certainly the entire area should be off limits to children as they are the most susceptible to the damages lead can do.

The other problem about scrounging for lead such as old roofing stuff or lead pipe is you may not know what is on/in it and what kind of byproducts the contaminantsmight produce. Best to melt such as that outside and stay up wind. I got a pretty good buzz (yeah, right! More like a dizzy headache) melting some lead pipe in my garage. Good clean lead ingots are far safer but sometimes ya gotta do.