From what I understand, in a pipelined CPU, every stage takes 1 cycle. But instructions are fetched from memory which takes up to ~150 cycles. The CPU fetches most instructions from the L1-cache, but I've read that it takes around 4 cycles. From that logic, a new instruction should only start every 4th cycle, which obviously makes no sense? enter image description here This diagram shows a new instruction starts every cycle.

So, how does the CPU fetch a new instruction every cycle, if it takes more than one cycle to even fetch an instruction?

  • 1
    \$\begingroup\$ There are different "cycles". There are many different clocks in the CPU, and the "main" CPU clock or the memory clock is not necessarily the same as instruction clock. \$\endgroup\$
    – Eugene Sh.
    Sep 15, 2021 at 16:56
  • \$\begingroup\$ Instructions are cached and prefetched. \$\endgroup\$
    – nanofarad
    Sep 15, 2021 at 17:05
  • 1
    \$\begingroup\$ More than 1 instruction will typically be fetched from L1 at a time. On a lot of systems, probably the whole 32 or 64 byte L1 cacheline is loaded, so you'll get 8 or 16 instructions fetched at a time, which is easily enough to hide a 4 cycle latency. \$\endgroup\$ Sep 15, 2021 at 18:10
  • \$\begingroup\$ @user1850479 Okay, I see. But only one instruction can be in the instruction register, right? Where is the rest of the loaded cacheline then stored? In some buffer with a single cycle access time? \$\endgroup\$
    – seb
    Sep 15, 2021 at 18:44
  • \$\begingroup\$ Where did you see L1 fetch takes 4 CPU cycles? \$\endgroup\$
    – jay
    Sep 15, 2021 at 19:23

1 Answer 1


It fetches the next instruction, so they are processed in parallel. Note that next instruction could also fetch the result of the instruction that isn't finished yet. Of course, this is unwanted feature, but it can be triggered by writing the assembler code by hand. The compiler has to keep track on how these instructions are processed and to use the result only after write output (WO).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.