0
\$\begingroup\$

There has been a fairly wide ranging discussion as to why it is difficult to combine memory / CPU logic on the same die (yield issue compounding, different processes, different clock-frequency, increasing testing requirements etc).

It seems that the latest NVDA GPU chips have >10GB of on-chip memory. These chips are also produced at leading edge nodes (i.e. 10nm below, TSMC process). So my question is: why does the memory / logic same die combination seem to work better for GPUs versus CPUs?

\$\endgroup\$
  • 1
    \$\begingroup\$ Are you sure? 10GB of on chip SRAM? \$\endgroup\$ – user3528438 Aug 20 '18 at 19:24
  • \$\begingroup\$ its DRAM for the GPU example. The tag is misleading \$\endgroup\$ – Daniel P Aug 20 '18 at 19:25
  • 3
    \$\begingroup\$ By on chip, you mean on board. For a GPU the memory is on the board, but it's not on the same chip as the actual GPU \$\endgroup\$ – Voltage Spike Aug 20 '18 at 19:31
5
\$\begingroup\$

It seems that the latest NVDA GPU chips have >10GB of on-chip memory.

On-package != on-chip. The HBM are stacks of several chips right next to the main GPU. Consumer grade GPUs have the memory soldered to the PCB in normal BGA packages.

GPU memory works "better" because there are no connectors that could impact the signal path between chips, allowing higher frequencies.

PC main memory uses sockets, this allows you to upgrade the RAM by changing or adding modules. But these also degrade the electrical signals, and thus performance.

Also note that only a few MB on the GPU chips itself are SRAM (caches) - the overwhelming majority of memory is DRAM.

\$\endgroup\$
2
\$\begingroup\$

It doesn't, here is a comparison of cache sizes:

enter image description here
Source: http://supercomputingblog.com/cuda/cuda-memory-and-cache-architecture/

A GPU pipeline looks like this:

enter image description here
Source: http://supercomputingblog.com/cuda/cuda-memory-and-cache-architecture/

A CPU pipeline looks like this:

enter image description here Source: https://microkerneldude.wordpress.com/2015/04/27/how-to-steal-encryption-keys-your-cloud-is-not-as-secure-as-you-may-think/

So why don't you see memory modules on a GPU PCB if they are the same? The first one I can think of would be that GPU's memory is much faster (and uses new memory technologies like DDR3 for a CPU vs DDR5 for a GPU) (look at the table). Memory module pricing is also likely an issue. It is much easier to put chips on a board than a module with connectors and size is also likely a constraint.

The actual size of memory is only limited by how much addressable space, so GPU's would probably win there, since CPU's generally have 64 bits and GPU's have more. What people use in practice is usually due to cost.

They are similar in that each one has an L1, L2 and L3 cache, then global memory. If the data isn't in the L1 cache, it looks in L2 which is bigger and slower, then in L3 which is bigger and slower than L2. If the data is not there it then searches in the global memory, and the processor has to wait a few clock cycles for this to happen.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.