Have a look at my answer to an earlier question, "How is binary converted to electrical signals?" for a bit of a philosophical take on this question. To turn that answer around a bit, we have a mental model of mathematics using binary symbols, and we design electronic circuits to do some physical process that we can interpret in terms of those binary symbols.
We can choose pretty much whatever physical quantities we want to represent those symbols. For example a current in a wire, a voltage on a wire, etc.
For storage, we can choose something very simple, like the position of a mechanical toggle switch. Of course that's very inconvenient, because it requires a person to intervene and physically move the switch whenever we want to change the stored value.
More commonly, we store symbols in the form of voltages on capacitors. There are dozens (at least) of ways to do this, depending on how long we need to store the data, how quickly we need to be able to access it, whether we can count on having a power supply available to maintain the storage, how much money we're willing to spend on it, etc.
If you want to understand how this can work on the physical level, one of the conceptually simplest forms of memory is the NOR Flash cell, shown here with voltages applied appropriate for the erasure process:
In this device, a charge is stored on the floating gate to represent a binary one or zero. The presence of the charge can be detected by its effect on the conductivity of the path between the source and drain terminals, much like a MOSFET. The main trick about this device is that because there's no wire connected to the floating gate, charge has to be driven there by hot electron injection, and removed by quantum tunnelling, which are processes that require some quantum physics background knowledge to understand.