There's laptops, PCs, microcontrollers and a lot of other things that can be plugged out and plugged in without a battery. But how does the system clock still keep track of time without anything to power it?
They use a small backup battery, which you can read about on Wikipedia:
Modern personal computer motherboards have a backup battery to run the real-time clock circuit and retain configuration memory while the system is turned off.
In computers this is usually called the "BIOS battery" and is generally a lithium cell such as a CR2032. In the photo below it is circled in red:
This is the same for portable devices which have their own battery, for example a laptop. You can prove this by removing the internal battery and seeing if it keeps the time.
So to answer your question they don't really keep time without power, they have a battery for it.
- connected devices ask for a time synchronisation as soon as they connect: computer usually gets it from the internet using NTP (network time protocol), GSM devices can also get it from their local cell tower.
- there are multiple services broadcasting time over the air waves: GPS is just a worldwide broadcasting system of an incredibly precise time. there are AM radio stations whose sole purpose is to broadcast their current time
- finally some smaller systems (microcontroller and embedded systems) do not really need an accurate time, just guarantee that time always flows in the same direction. They would save their timestamp every now and then in a permanent memory area and restart from their last known time on next start.
Without a power source an electronic device can't keep track of time. Batteries are power sources so they can't be thought of as being able to keep time without power.
There is another strategy that doesn't keep track of time. When you 'sync' your device with the computer time on the device is synchronized.
Besides the "big flat battery on a computer motherboard", there is another device other answers don't mention: the RTC (real time clock).
Some microcontrollers do have such a feature, by having an internal battery inside the chip, which can keep the time for decades without external power.