14
\$\begingroup\$

I'm trying to understand why certain CPU cache memories are faster than others. When comparing cache memory to something like main memory, there are differences in memory type (SRAM vs DRAM), and locality issues (on-chip vs having to traverse a memory bus) that can affect access speeds. But L1 and L2 are generally on the same chip, or at least on the same die, and I think they are the same type of memory. So why is L1 faster?

\$\endgroup\$
  • 9
    \$\begingroup\$ Because it's built that way. \$\endgroup\$ – Ignacio Vazquez-Abrams Mar 31 '14 at 1:54
  • \$\begingroup\$ There are multiple ways of building the same type of RAM. Some ways result in RAM that is faster than RAM built in other ways. \$\endgroup\$ – Ignacio Vazquez-Abrams Mar 31 '14 at 2:53
  • \$\begingroup\$ The same reason why your DDR RAM faster than your HDD... \$\endgroup\$ – hassan789 Mar 31 '14 at 2:56
  • 1
    \$\begingroup\$ related: my answer on and SO question about cache sizes also explains some speed / power-consumption tradeoff reasons for having multi-level caches. Basically, making a cache super-fast costs power and die-area to do more in parallel, and is incompatible with the large sizes / associativity that you want in a last-level cache. \$\endgroup\$ – Peter Cordes Jul 24 '16 at 9:27
24
\$\begingroup\$

No, they're not the same type of RAM, even though they're on the same chip that uses the same manufacturing process.

Of all the caches, the L1 cache needs to have the fastest possible access time (lowest latency), versus how much capacity it needs to have in order to provide an adequate "hit" rate. Therefore, it is built using larger transistors and wider metal tracks, trading off space and power for speed. The higher-level caches need to have higher capacities, but can afford to be slower, so they use smaller transistors that are packed more tightly.

\$\endgroup\$
  • 3
    \$\begingroup\$ This is what I'm looking for, thanks. Do you know any good sources where I can read further about the differences? \$\endgroup\$ – ConditionRacer Mar 31 '14 at 3:59
  • \$\begingroup\$ No, not offhand. The details of these space/power/time tradeoffs, as well as choices regarding cache capacity and architecture, are tied very closely to the proprietary details of the manufacturer's processes, so very little (if anything) is published about it. I can only speak in general terms. \$\endgroup\$ – Dave Tweed Mar 31 '14 at 4:16
  • 10
    \$\begingroup\$ L1 is also smaller, meaning signal propagation across it takes fewer clock cycles : often the size of L1 is decided as the largest memory accessible in 1 (or 2) clocks. \$\endgroup\$ – Brian Drummond Mar 31 '14 at 10:22
  • 6
    \$\begingroup\$ In addition to Brian Drummond's point, L1 also typically uses data-tag parallel access. L2 typically accesses tags first (determining which way if any hit) then data, increasing latency but saving energy (significant given larger size, higher associativity, and higher miss rate). (L2 access also typically begins after an L1 miss is confirmed which increases it effective latency.) \$\endgroup\$ – Paul A. Clayton Mar 31 '14 at 12:09
  • \$\begingroup\$ @BrianDrummond - smaller both in terms of wire-load, and gate depth (I don't think that is obvious from your comment). \$\endgroup\$ – Sean Houlihane Apr 13 '17 at 22:02
5
\$\begingroup\$

L1 is usually used as a storage for decoded instructions, while L2 is a general cache for a single core. The lower the cache the smaller size it is and faster it usually is. As a rough rule of thumb for PC processors:

L1 Cache: 2-3 clock cycle access

L2 Cache: ~10 clock cycle access

L3 Cache: ~20-30 clock cycle access

The design of the L1 cache should be to maximize the hit rate (the probability of the desired instruction address or data address being in the cache) while keeping the cache latency as low as possible. Intel uses an L1 cache with a latency of 3 cycles. The L2 cache is shared between one or more L1 caches and is often much, much larger. Whereas the L1 cache is designed to maximize the hit rate, the L2 cache is designed to minimize the miss penalty (the delay incurred when an L1 miss happens). For chips that have L3 caches, the purpose is specific to the design of the chip. For Intel, L3 caches first made their appearance in 4 way multi-processor systems (Pentium 4 Xeon MP processors) in 2002. L3 caches in this sense greatly reduced delays in multi-threaded environments and took a load off the FSB. At the time, L3 caches were still dedicated to each single core processor until Intel Dual-Core Xeon processors became available in 2006. In 2009, L3 caches became a mainstay of the Nehalem microprocessors on desktop and multi-socket server systems.

Quote sourced here from "Pinhedd's" response.

\$\endgroup\$
  • 1
    \$\begingroup\$ L1 is used for data as well generally but it's true that often L1 data and L1 instruction are separate while L2 is shared: programmers.stackexchange.com/questions/44731/… (one reason why it can be shared is that it has more ways). \$\endgroup\$ – Guy Sirton Mar 31 '14 at 22:01
  • 1
    \$\begingroup\$ It is common for processors to have separate instruction and data caches at the L1 level. Certainly true of later x86 and SPARC chips. And some L1 instruction caches don't store decoded instructions. The Pentium 4 stored decoded instructions, but then Intel went back to regular I-caches, then recently added decoded instruction caches back into their chips. \$\endgroup\$ – Craig S. Anderson Nov 9 '14 at 10:12
3
\$\begingroup\$

There are several reasons why speed is inversely proportional to size. The first that comes to mind is the physical domination of conductors, where signal propagation i limited to some factor from speed of light. An operation may take as long as it will take an electrical signal to travel the longest distance inside the memory tile and back. Another related reason is the separation of clock domains. Each CPU runs off its own clock generator, which allows the CPU to run on multi-GHz clocks. Level-1 cache runs at and is synced with the CPU clock, which is the fastest in the system. Level-2 cache on the other hand has to serve many CPUs, and is running in a different (slower) clock domain. Not only the L2 clock slower (larger tile) but to cross a clock domain boundary adds another delay. Then of course there are the fan-out issues (already mentioned).

\$\endgroup\$
1
\$\begingroup\$

Apart from inherent performance characteristics, locality also has a role (L1 is closer to the CPU). According to What every programmer should know about memory:

It is interesting to note that for the on-die L2 cache a large part (probably even the majority) of the access time is caused by wire delays. This is a physical limitation which can only get worse with increasing cache sizes. Only process shrinking (for instance, going from 60nm for Merom to 45nm for Penryn in Intel’s lineup) can improve those numbers.

\$\endgroup\$
  • \$\begingroup\$ I think this applies mostly where the caches are large - This doesn't apply to all processor types (although the wire delay is significant). \$\endgroup\$ – Sean Houlihane Apr 13 '17 at 22:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.