No electronics runs on mains power, it is all running from a DC power supply of some sort. A power indicator LED needs to be simple and reliable, the easiest way to do this is an LED and resistor connected across the output of that DC power supply.
In order to avoid the output voltage of a power supply from changing significantly if the load changes and to reduce noise in it's output all DC power supplies will have some fairly large capacitors on their output that act as a short term power reserve.
When you turn something off that DC power supply will turn off and stop supplying power but so will most of the things using power. e.g any displays / backlights / LEDs under control of the electronics will normally be turned off and processors will normally get put into a halt or sleep mode rather than just waiting for them to crash due to a low voltage. So you end up with the power supplies output capacitors being fully charged but there being hardly anything other than the power LED using up that power.
Since the power LED doesn't use up much power the end result is a power LED that remains on for a reasonable amount of time as the internal voltage slowly drops.
So the LED isn't remaining on after power has been removed, it is a fairly true and fast acting (a lot faster than the response time of the human eye) indicator of the internal voltage. What you are seeing is the internal power taking a while fade away after external power is removed.
You won't notice this sort of thing nearly as much with something like incandescent bulb power indicators because they take a lot more power and so the power indicator itself will drain the remaining power very quickly.
 Depending on exactly where the off switch is in the system the power supply may carry on running for a short while. The supply will also have input capacitors, if the off switch disconnects the electricity into the power supply rather than signalling the supply to switch off its output then it will keep running until those input capacitors are drained. You can sometimes guess at whats going on based on the brightness of the power LED, does it stay full brightness, fade out or stay full brightness for a while and then fade out.