The core of basic computer is comprised of "binary" elements, or flip-flops. Or FF for short. The element has two states, "flip-left", and "flip-right". When the FF flops, "electricity flows" from one side to another in a fast, avalanche-like process, one side assumes high potential (voltage), the other assumes "low" voltage. Either side of FF can be considered as "right output", and all FFs are following this arbitrary convention.
As result, an information gets represented as "low voltage", and "high voltage", both somewhere in between the ground and supplying voltage. It is important to note that these two levels are fairly distinct, they don't have a continuous "analogish" spread of values. That determines the fundamental difference between analog and digital electronics. And the FF can stay in this state as long as the supply voltage is there, unless a special kick is applied so the FF can flip. These "storage elements" can be implemented in several different ways, but the principle is the same - their output assumes either a distinctive "high" level, or "low", with a noticeable gap between these two levels.
Once the levels are defined and distinct, there are elements that can discriminate between these two levels, just like a normal comparator. If a signal is above some threshold between "low" and "high", the result is amplified to "strong high". This would be a simple logical buffer. One can say that the output has "assigned" high level, or "1". Since the power supply usually is limited to certain level, 1.8V, 3.3V, or older 5V, the actual output voltage doesn't go too far, and stays within the same voltage "bin" as in the original flip-flop. So we have a consistency in voltage levels representing "low" and "high", or "0s" and "1s".
If the output gets amplified in opposite direction, it will be called "inverter". The next in complexity is a "gate" that receives two input signals, so a certain combination of them results in "strong output". Look up NOR gate as a fundamental example.
So, a computer logic "knows" which is "0" and which is "1" by sensing the difference between two levels (say, with "buffers" as described above), and flipping other internal FFs into corresponding states if needed/instructed. The actual FFs are a bit more complicated than the one shown at the beginning of Wikipedia article, scroll down to more useful FFs called D-flops. They have "reset" signal allowing to put every FF into (known) initial state, and have "enable" inputs that allow to distribute/latch common signals with discretion.
The rest of a computer is simple - FFs are grouped into registers that are wired to buses of buffers, some functions (a bit more complex than NOR) are designed, clock is ticking, and when instruction codes are fetched from memory (similar kind of flip-flop arrays), sequences of logic operations lead to desired results.
Does this explanation address your concern?