The short answer for low core voltage in IC's: obtaining improved power, speed, density and cost made possible by ever-shrinking process nodes requires a co-dependent relationship with core voltage.
First, voltage vs. power.
There's two types of power dissipated in an IC:
- static (leakage) power: residual current due to incomplete FET turn-off
- dynamic (switching) power: current used to charge and discharge downstream loads
Static power is pretty easy to understand. It is modeled as a resistance as all the p- and n-FET pairs leaking in parallel, and so its power is simply \$P = E^2/R\$. The lower the voltage then, the lower the leakage power, squared.
Dynamic power takes into account switching frequency, capacitance and voltage, as \$P = 1/2CV^2\$. \$C\$ is calculated from the summed capacitance of all the driven signals as they charge and discharge their loads. Again, note the squared power-to-voltage relationship.
So even before you get into fiddling with process, reducing Vdd voltage is a win (squared) for power.
Now, process vs. voltage vs. speed.
Deep-submicron logic employs transistors that use thinner oxides and shorter channel lengths. Thinner oxide allows lower threshold voltage for faster switching, and shorter channel also reduces switching time. Yay, more speed!
But, thinner oxide means lower gate-source breakdown voltage. Shorter channel length meanwhile reduces source-drain breakdown voltage. Low threshold FETs also leak more.
Which brings us back to that co-dependency between deep-submicron logic and low Vdd. In fact, small-geometry FETs with thin oxide, low-threshold gates would be impossible to use without also reducing Vdd.
But we're not done yet. Deep-submicron technology is infamous for a major problem: leakage. This is mitigated by advances such as Silicon-on-Insulator and FinFET technology, which keep static power consumption in check, if not reduced outright.
The news gets better on the dynamic side: smaller geometry has less driven capacitance, which decreases \$1/2fVC\$ power at a given frequency.
The result? Smaller-process node ICs actually use less power overall than their equivalent in a bigger process node. A rule of thumb is that for every halving of channel length you net about 1/3 power reduction, before accounting for SoI or FinFET adoption.
So, another double-header win (power, and speed) for lower voltage.
That's great for the core, but what about the rest of the system?
Most bigger ICs split the difference by using multiple supplies: low-voltage cores, and higher voltage I/O. The I/O is handled by bigger, thick-oxide FETs in the near-pad circuits that can withstand the higher I/O voltages needed by the rest of the system. This allows the low-voltage optimized core to work at its best, while maintaining I/O compatibility with support chips.
There is a penalty for mixed voltages: extra fab steps to make the thick oxide FETs, more area set aside for both the higher-voltage transistors and the translation to/from core, and of course more power pins. All of which increase wafer cost (and the pins, package cost.)
Nevertheless, for a core-limited die, by using a smaller process you get all of that cost back - and more - by reducing your die size. You get more chips per wafer, even if the wafers themselves are more expensive than a larger process node.
The takeaway: reducing logic voltage helps power, while enabling smaller process geometry which in turn enables greater logic density, higher speed, and lower cost.
It's not an overstatement to say that today's paradigm-shifting electronic products we take for granted, like smartphones and broadband, would have been impossible without chip designers taking maximum advantage of process technology. The seeming 'annoyance' of multiple supplies is a small price to pay for this incredible progress.
Thus, as process has evolved, so we have witnessed core voltage migrate from 5, to 3.3, to 1.8, to 1.5, to 1.2, to 1.0, to 0.8, and lately (2023), to 0.6V for advanced 5nm processes.
By now I've hopefully convinced you of the value of using low voltage IC core supplies. Now, why the continued use of higher voltages in systems?
System power is a different problem set than an IC. In systems, we're dealing with multiple loads that are physically separated. Because of this, it's often better to increase the voltage and thus decrease the current being shipped around to the loads, allowing smaller wires and reducing \$P = I^2R\$ wire losses, then convert the voltage locally at the load as needed (this is called Point of Load regulation.)
For example, a GPU board will be supplied with 12V from its slot and cabled power connections, which gets converted locally to very high current (100's of A) 1.0-ish V core supply. Point-of-load regulation deals with this very effectively. Otherwise it would be impractical to power the GPU if it were obliged to run on 12V (see above), or if the motherboard had to supply 100A at 1.0V-ish over cabling with almost no IR drop.
Another example, LED strings with LED wired in series may use 24, 36, or 48V, current-regulated. Wiring and driving this way is not only more efficient than running them in parallel, but also ensures consistent brightness between LEDs as they're all seeing the same current.
Taken to an extreme, we see ever-higher voltages being used for electric vehicles (800V in the Porsche Taycan for example): it's easier to stuff more volts down the wire into the motor coils than it is to upsize the wires carrying the current. This practicality is also why the bigger (Level 3) EV chargers use 480V 3-phase power instead of 120 or 240V.
We're fortunate that we live in an age where power electronics have evolved to the point where we can efficiently optimize the voltage / current tradeoff to make the most of the system using it. New developments in power supply design, such as GaN and SiC devices are already making an impact in handling ever-higher voltages and currents, and advanced poly-phase switchers deal with power-hungry GPUs, CPUs and other big chips.
In other words, today we have a whole power ecosystem that allows us to tailor the voltage to the load, rather than the load to the voltage.
This wasn't the case in the mid-1970s when I started out in electronics. Then, 5V TTL was still regarded as state-of-the-art, while switching regulators were considered exotic and costly.
Still, 5V persists today as a kind of 'Goldilocks' voltage that is high enough to do useful hobby-tronics stuff like run motors, blink lights and power radios; yet low enough to not result in ridiculous power consumption for small systems.