I have had the following idea / opinion in a long time, but I am not sure if it is true.
From what I understand, in a processor, coarsely the voltage
U required to operate is proportional to the frequency
f at which it is run, and the computing power is proportional to the number
N of transistors present and the frequency
Therefore, one could:
- divide the frequency of the processor by
k, allowing to divide the voltage by
- increase the number of transistors by
This would in turn give a computing power increased by a factor
k (increased by
k^2 because of the number of transistors, decreased by
k because of the frequency), for an electric power unchanged (power is
U^2/ R * N where
R is the electric resistance,
U is divided by
N is increased by
k^2). If Moore law gives more and more transistors for the same price, then you should have no limit to the computing power you achieve (at the cost of reduced frequency and possibly requiring parallel hardware and algorithms).
Is that reasonable (or even true), or is there an underlying error? Naively / loosely, I think about this as an explication of why the brain is so much powerful while using a bit less energy than a CPU (brain is typically 20 watts and 100Hz, CPUs nowadays often from 35 to 130 watts and 3GHz, some people say ).
Yes, I know that the energy consumption / power consumption is the wall that processors are meeting. Here I talk in terms of voltage (before converting to power consumption) because it is what predicts (or so I believe?) which frequency one can run at.
The computing power does is proportional to the number of transistors. For example, one can simply build more cores given more transistors. The problem then is power consumption; this is why I consider a decrease of frequency, so that power consumption increase by increasing the number of transistors is zeroed by the power consumption decrease due to running at lower voltage (and, hence, frequency).
I know that this will not increase the one thread execution speed, and will require parallel algorithms, but this is not the question. In the same way, architecture is not the question either. I am aware that manufacturers now add more and more transistors to the caches etc to increase one thread execution speed / reduce latency, but this is not what I ask for here. Here I ask only about whether a very general scaling argument is true, then using this scaling with parallel software is another question.
By the way, we are getting better at using parallel architectures: Artificial Neural Networks on GPUs are all about it. This is exactly the idea behind the brain also: very slow at one thread operation, but incredibly parallel and powerful computing power. What I want to understand really is: given the silicium technology used on current transistors, can we in theory if Moore law holds (i.e., we get more and more transistors for the same cost) build something as powerful as the brain that does not use megawatts or more. For this, it sounds that the solution is to increase parallelism, and reduce frequency (as in the brain). For example, if my scaling argument holds, you can get the frequency of your chip from 3GHz to 100Hz (i.e. divide frequency by
alpha= 30 millions) and by adding a LOT of transistors (
alpha^2, but if you expect Moore law to hold, we will get it ultimately), and therefore increase the computing power of the chip by the same 30 millions factor. I agree, this is not that straightforward to pack so many transistors, maybe you would need a 3D chip (like the brain) or another architecture change, but I am just interested about the scaling.