A capacitor holds its charge when the battery is removed because there is no longer a closed circuit to allow electrons to move to balance the charge out.
You can think of it with water analogies. You have a pump (battery), tubes (wire), stopcock valves (to simulate opening the circuit), and a rubber membrane that flexes when water on one side of the membrane is at a higher pressure than the other side (capacitor).
To begin the valves are open and the pump creates pressure (voltage) on one side of the closed loop. That pressure difference causes the rubber membrane to flex. Close both valves and then remove the pump from the loop. Once the valves close, the pressure on each side of the membrane remains the same whether or not the pump is attached. The rubber membrane will therefore be in a flexed state. The energy storage of a capacitor is equivalent to the energy stored by the rubber in a stretched state. If one were to open both valves after this, it would spurt out a bit of water at pressure until the rubber membrane relaxes.
Simplistic diagram:
pump ---> valve ---> rubber membrane
^<--------- valve <--------
Specifically to the electrons in a capacitor issue, you have to remember that electrons are everywhere already. When the battery is connected, the entire wire from negative terminal to the negative plate of the capacitor is now at a single potential. That means all of those electrons are at the same "pressure". If you open the circuit, the extra electrons at the negative plate have the same pressure/voltage as the electrons in the wire that you just disconnected from the negative terminal of the battery. Since all of those electrons are at the same voltage, they balance each other out and have no desire to move away from the capacitor plate because then they'd be squeezed together even more in the wire.
The reason the water analogy works is because an open circuit isn't a tube without water waiting to be filled. It's a tube already full of water that has a stopcock on it preventing water from moving in either direction.