I remember my first experience with electricity in school was when we got a bench power supply, a potentiometer, and a few wires, and were invited to experiment. We learned that for a few configurations nothing happened (very educational!), but if we connected it the right way we saw the current meter go up and down when we turned the potmeter's knob. We didn't understand yet what actually happened, but we felt like magicians. Later the teacher explained about current and resistance.
So that's one way to get started. You need a decent, current limited, power supply, so that the experiments can't do any harm. A 9V battery may be an alternative. Let the boy play with small lamps (LEDs are more vulnerable) and switches, and teach him about parallel and series switches. Let him construct a buzzer with a relay.
He'll be most interested and eager to learn if he can see or hear something happen.
edit
I wouldn't expose the kid to electronic parts, not even a diode, until he fully understands voltage and current. A diode may look simple to explain, but it's no use explaining it if you can see him think: "Yes, current... what was that again?"