You are talking about the circuit running around with the name, Joule Thief, and various related incarnations of the same. Your behavioral schematic is pretty close. You don't need the diode there if the load is an LED. (In a typical Joule Thief circuit, a BJT provides the switch but also requires a driving circuit which is what slightly complicates the schematic.)
Sticking with the behavioral schematic: Without the switch closed, the battery source, the coil, and the LED are in series. But no appreciable current flows because the LED requires a substantial voltage in order for any useful current to flow. There may be a tiny current that is too small to matter. But that's the circumstance without an active switch toggling on and off.
When the switch closes, the current builds up in the coil according to the usual formula for that:
$$\frac{\textrm{d} I}{\textrm{d} t}=\frac{V_t}{L}$$
With \$V_t\$ being the battery voltage (which if this is a real battery will probably vary a little as the load changes over time.) As the current increases, so does the energy stored in the magnetic field. When the switch opens, the direction of the current through the coil continues, but because the current is now declining, the sign of the voltage across it changes. (Just look at the above equation and imagine that \$\frac{\textrm{d} I}{\textrm{d} t}\$ changes from increasing (positive) to decreasing (negative.) You should be able to see why the sign of \$V_t\$ must therefore change. (It will change enough so that the LED will be forward biased enough to conduct the necessary current.)
The current will continue to flow through the LED, but it will now be declining towards zero. If the switch is left open long enough, the current will decline to zero. This would be the right time to engage the switch again and start the cycle all over again.
The Joule Thief itself uses a particularly oriented winding in the BJT base circuit so that it aids the battery voltage and drives harder on the base while the other side of that transformer (a coil, too) in the collector leg allows the current in it to rise up. Eventually, one of two things stops this process: (1) the BJT's \$\beta\$ gives out and the BJT's collector voltage starts to rise, triggering a reversal; or, (2) the transformer core saturates, causing the collector current to suddenly rise beyond what the base can support (again exhausting the BJT's \$\beta\$) and causing the BJT's collector voltage to start to rise and triggering a reversal. Either way, once that process starts, the rising current cannot any longer be supported and is forced to start declining. And when it declines, the voltage on the coils reverse, which causes the coil in the base portion to also reverse and now oppose the battery, pretty much forcing the BJT to turn off (until the cycle can reverse, when the magnetic energy has expired.)