6
\$\begingroup\$

enter image description here

Processors use both data and instruction caches in order to reduce the number of slow accesses to main memory. However, while it is clear to me that the data cache's purpose is to store frequently used data items (such as elements in an array or inside a loop), I cannot see what exactly the instruction cache stores that helps alleviate memory access times.

In the image above, we have an example of an "addi" instruction which adds a constant value to the value stored in general purpose register "r2" and writes the result to general purpose register "r1".

After this instruction is executed, what exactly is saved to the cache?

  • It can't just be the opcode - most CPU instruction sets contain just a few hundred unique opcodes or less, so if the instruction cache was pre-loaded with all possible opcodes, it would always have a 100% hit rate. However, that would defeat the purpose of having a cache, plus I've read that instruction cache misses are very much possible.
  • It can't be the values from main memory which are loaded into the general purpose registers, since that's exactly what the data cache is for.

Thank you in advance.

\$\endgroup\$
1
  • \$\begingroup\$ Why do you think it matters to the cache if a particular instruction was executed or not? Instructions usually don't change at runtime. \$\endgroup\$
    – Dmitry Grigoryev
    Commented May 14, 2019 at 6:50

3 Answers 3

Reset to default
21
\$\begingroup\$

It literally stores lines of machine code from program memory (aka the entire instruction you line in your original post.

The fact you even discuss "storing all possible op codes in cache" points to a deeper misunderstanding. Talking about storing all possible op codes in cache (or any memory for that matter) has no meaning. All the possible opcodes that the processor can run are hard-wired into the logic circuitry of the processor. They aren't "stored" anywhere.

\$\endgroup\$
8
  • 3
    \$\begingroup\$ Note that this is only true for most CPUs. Recent Intel x86s store decoded micro-operations (ie. the output of an early stage of the execution process), and I think AMD may have also switched to a micro-op cache rather than a strict instruction cache. \$\endgroup\$
    – Mark
    Commented May 13, 2019 at 20:29
  • 2
    \$\begingroup\$ @MartinX When you say "the entire instructions were somehow hard wired" are you saying that you thought something like entire "ADD, Reg1, Reg2" was hardwired? And then something like "ADD, Reg2, Reg3" was a separate hard-wiring? Because that's not the case. Not every possible combination of opcode and argument has a unique circuitry hard-wired into the CPU.. \$\endgroup\$
    – DKNguyen
    Commented May 13, 2019 at 21:50
  • 7
    \$\begingroup\$ @Mark: Intel P4 had a trace cache instead of an L1i cache. This worked out badly and was a big bottleneck (because it was slow to build traces on misses with its weak decoders). Intel since Sandybridge (realworldtech.com/sandy-bridge) and AMD since Zen still have regular L1i caches that cache x86 machine code bytes, but also have smaller very fast decoded-uop caches. They still have powerful decoders for good throughput on uop cache misses, and it's not a trace cache. (A uop cache line can only cache contiguous uops from one 32B chunk, instead of following jumps.) \$\endgroup\$
    – Peter Cordes
    Commented May 13, 2019 at 22:04
  • 6
    \$\begingroup\$ @Mark: Some older AMD CPUs do store extra metadata alongside the L1i cache: they mark instruction boundaries in the cache to speed up decode. See Agner Fog's microarch pdf. Also David Kanter mentions the pre-decode metadata in realworldtech.com/bulldozer/4. More info about it in his K10 write-up: realworldtech.com/barcelona/4 \$\endgroup\$
    – Peter Cordes
    Commented May 13, 2019 at 22:10
  • 3
    \$\begingroup\$ @Toor: Intel calls their decoded-uop cache the "Decode Stream Buffer (DSB)" including in HW perf counter event names. Physically, its very much built as an associative cache with each "way" of a set holding up to 6 uops. It's indexed and tagged by virtual address (so it bypasses TLB lookups). Of course caches are built out of "tightly coupled" SRAM arrays, but what makes them caches is the management system and the lookup / indexing mechanism. \$\endgroup\$
    – Peter Cordes
    Commented May 13, 2019 at 22:16
13
\$\begingroup\$

The Instruction cache stores the most recently used instructions and their addresses so that if an instruction needs to be repeated it doesn't have to be retrieved from main memory - this is much quicker.

For example the first time a loop is performed the instructions will be retrieved from main memory and simultaneously placed into the cache. On subsequent iterations of the loop the instructions can then be quickly retrieved from the fast cache memory.

The addresses are stored in the cache together with information that indicates whether the cache is up-to-date so the CPU control knows whether it can use the cached instructions or needs to go to main memory.

\$\endgroup\$
1
  • 5
    \$\begingroup\$ Good answer. May be worth emphasizing that the instructions are placed into cache as they are retrieved from memory (and indeed, before they are executed) to clear up the OP's potential misunderstanding that the instruction is saved to cache "after [it] is executed." \$\endgroup\$
    – Shamtam
    Commented May 13, 2019 at 17:07
9
\$\begingroup\$

The instruction cache stores the individual instructions for the CPU of the currently executing program. It is the program itself. Main memory is often too slow (or has too much latency) to be able to feed the CPU its next instruction every time it is ready for one. This is why a fast cache near the CPU is used, this is the instruction cache.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.