It's good practice to avoid eating electronic components.
It's also good practice to wash your hands after handling electronic components, or anything for that matter, before handling food that you, or your family, are going to eat.
As long as you follow those two good practices, there is essentially no health difference to you and your family whether your components RoHS or not, or whether you're using lead-bearing solder or not.
As a number of comments have pointed out, don't breathe the fumes from soldering. It's not the metals, it's the burning flux that matters. Depending on how much soldering you do, solutions from a fume extractor to a fume mask may work. I do so little soldering that I avoid breathing in the fumes by breathing out slowly and continuously while making each joint. This stops me ingesting the hot concentrated smoke from the iron, but doesn't protect me from any cold, diluted fumes in my ventilated room. I'm not recommending this method to anybody else, just saying what I do.