Electrical Engineering Stack Exchange is a question and answer site for electronics and electrical engineering professionals, students, and enthusiasts. It only takes a minute to sign up.
Sign up to join this community
There has been a fairly wide ranging discussion as to why it is difficult to combine memory / CPU logic on the same die (yield issue compounding, different processes, different clock-frequency, increasing testing requirements etc).
It seems that the latest NVDA GPU chips have >10GB of on-chip memory. These chips are also produced at leading edge nodes (i.e. 10nm below, TSMC process). So my question is: why does the memory / logic same die combination seem to work better for GPUs versus CPUs?
It seems that the latest NVDA GPU chips have >10GB of on-chip memory.
On-package != on-chip. The HBM are stacks of several chips right next to the main GPU. Consumer grade GPUs have the memory soldered to the PCB in normal BGA packages.
GPU memory works "better" because there are no connectors that could impact the signal path between chips, allowing higher frequencies.
PC main memory uses sockets, this allows you to upgrade the RAM by changing or adding modules. But these also degrade the electrical signals, and thus performance.
Also note that only a few MB on the GPU chips itself are SRAM (caches) - the overwhelming majority of memory is DRAM.
It doesn't, here is a comparison of cache sizes:
Source: http://supercomputingblog.com/cuda/cuda-memory-and-cache-architecture/
A GPU pipeline looks like this:
Source: http://supercomputingblog.com/cuda/cuda-memory-and-cache-architecture/
A CPU pipeline looks like this:
So why don't you see memory modules on a GPU PCB if they are the same? The first one I can think of would be that GPU's memory is much faster (and uses new memory technologies like DDR3 for a CPU vs DDR5 for a GPU) (look at the table). Memory module pricing is also likely an issue. It is much easier to put chips on a board than a module with connectors and size is also likely a constraint.
The actual size of memory is only limited by how much addressable space, so GPU's would probably win there, since CPU's generally have 64 bits and GPU's have more. What people use in practice is usually due to cost.
They are similar in that each one has an L1, L2 and L3 cache, then global memory. If the data isn't in the L1 cache, it looks in L2 which is bigger and slower, then in L3 which is bigger and slower than L2. If the data is not there it then searches in the global memory, and the processor has to wait a few clock cycles for this to happen.
