Physics explanation
Light bulbs
Incandescent light isn't really a light source so much as a heating element. Any current through a wire heats it up a bit (Joule's Law of Heating); once the wire is above room temperature it emits net energy via black-body radiation. The rate at which this energy is emitted depends on the fourth power of temperature, i.e. the higher the temperature the brighter†. And the more current (or equivalently‡ more voltage), the higher the temperature of the wire.
The fundamental physical process behind blackbody light emission is this: the atoms in a warm piece of material are shaken around by thermal motion. This motion is completely chaotic, so even if the average energy per atom is rather low, every once in a while an atom at the surface will get a push from multiple neighbours and thus gather enough energy that it can emit a visible photon (at least \$2.6\times10^{-19}\$ Joules). But far more often, it will only have enough energy to emit an invisible infrared photon.
LEDs
By contrast, LEDs pump the atoms directly to the energy that's required for emitting visible light. They do this by cleverly exploiting the band gap of a semiconductor. That's a quantum-mechanical feature of crystals like silicon, which “forbids” electrons from having energies in a certain range. You then take one piece of semiconductor which has been doped so the conductance electrons are all above the band gap, and one where they're all below the bandgap. Then, when a current flows across the junction, each electron loses just the right amount of energy to excite an atom to produce a photon with the right energy to be visible – again, for red light, this is about \$2.6\ldots3.2\times10^{-19}\$ Joules.
Only... why would the electrons continue to go over the junction? After an electron has crossed the junction, it won't be inclined to climb across the band gap again; that costs energy which the electron doesn't have.
...Unless you give it the energy from an external source: each volt that you apply to a circuit can supply an electron with an energy of \$1.6\times10^{-19}\:\mathrm{J}\$, a quantity that physicists call just an electron volt. So when you apply a voltage of \$U\$ to an LED whose band gap has an energy of \$U\times1\:\mathrm{eV}\$, you can keep up a current. This voltage isn't really dependent on how much current actually passes through the LED, therefore the brightness can't effectively be regulated by tweaking the voltage – you need to regulate the current instead. And if the voltage falls below the band gap, the current will just cease entirely, because the conductance electrons just won't go into the n-doped domain at all anymore.
†That's a bit too simplistic: Stefan-Boltzmann describes the intensity integrated over the entire electromagnetic spectrum. Only a narrow band of that is actually visible (that's the reason incandescent light is so much less efficient than LEDs). Since the wavelength of peak intensity also depends on the temperature, brightness is in fact related not just to \$T^4\$ but to a more complicated relation, but still: higher temperatures always correspond to brighter light.
‡Similarly, Ohm's law isn't completely correct here because the resistivity depends on temperature. But the qualitative dependency higher voltage ⇒ higher electrical power still holds true.