This isn't how the physics works, but from a circuit point of view you can think of a inductor as having current inertia. The bigger the inductor, the more inertia the current has.
When you apply a fixed voltage accross a inductor, the current builds up linearly. If you were then to short out the inductor so the current could circulate, it would do so forever if the inductor were perfect. Real inductors you can buy are made from wire, so have some finite resistance. The current times that resistance builds up a reverse voltage that slows down the current. But since the reverse push is proportional to the current, not fixed, the current decays exponentially instead of in a linear ramp if the current was fixed.
Actually inductors have been made from superconducting material, and they really do circulate current forever if the whole loop is superconducting.
If you can picture a inductor providing inertia to current and therefore how a fixed voltage causes the current to linearly ramp up, it's time to consider what happens when someone tries to suddenly interrupt that current. Think of trying to instantly stop a moving mass. Two things will happen. First, it won't stop instantly. Second the mass will create a great deal of force against whatever is trying to stop it. The inductor will do the same, but here force is voltage. The faster you try to stop the current, the more the inductor will push back with higher voltage.
But you say, a switch stops the current instantly when opened. Even if a switch were perfect and could do that, there would still be some point at which the contacts just barely separated. The inductor doesn't have to create much voltage for the current to arc between the contacts. Once a arc is formed, it's easier to keep it going at greater distances. That's because the air you see light up as a spark has become a plasma, which conducts electricity fairly well. So the switch contacts may have separated, but are now still connected by a plasma arc "wire". It does take some voltage to keep this arc going, which pushes backwards against the inductor current, which causes the current to decrease.
Eventually there won't be enough current to keep the arc going, and the switch is finally completely open. At that point most of the energy stored in the inductor has been spent, and the little that's left charges up the inevitable parasitic capacitance that always exists accross the inductor. Now you have a L-C tank circuit that will oscillate back and forth for a while. The little remaining energy is dissipated by the resistance of the wire in the inductor as the current sloshes back and forth thru it. The oscillations die down, and everything is finally truly off to the extent you can measure or care about.
This arcing accross switches is very real and a problem for switches and relays. This is one reasons relays wear out and often have different ratings for inductive loads. Every arc will damage the switch a little bit, which is considered in the lifetime cycles rating of the switch or relay.
Transistors can also be used to switch off inductors quickly. In fact, this is the basis for the common boost converter switching power supply topology. By charging up a inductor with current and then deliberately trying to switch it off quickly, you can harness the fact that the inductor will make a higher voltage for you than you started with.
