On understanding tradeoffs associated with pipeline depth

Question

In Weste and Harris's CMOS VLSI Design, they write the following in the context of a discussion about how different levels of design abstraction interact but, to be clear, my question is about the specific claims made and so it's more of a computer organization question:

Deeper pipelines allow higher frequencies but consume more power and lead to greater performance penalties when operations early in the pipe- line are dependent on those late in the pipeline. Many functions have various logic and circuit designs trading speed for area, power, and design effort.

My questions are listed below (and in brackets I have put my best guess to show that I have thought about each):

(1) What is meant by deeper pipelines?

(Does this mean more pipeline stages?)

(2) Why do they allow for higher clock frequencies

(Is this because the combinational circuits between each stage are smaller and therefore we have less propagation delay, and so need less cycle time before each pipeline register needs to setup?)

(3) Given a deeper pipeline, why does it follow that they use more power?

(Is this because of the higher frequency, or is it that even at the same frequency as a less deep pipeline they will tend to use more energy because they are clocked into extra registers which switch, costing dynamic power?)

(4) Why does a less deep pipeline not have the same performance issues if there are dependencies between operations in the pipeline?

(Not really sure at all here. Is it that in a deeper pipeline the dependant operation would have to idle or something? Don’t we have pass back strategies to avoid this? So why does depth of pipeline matter)?

Answer 1

Your quote is from [Weste2011], Section 1.6.1 on "Design Abstractions", pp. 30.

Your questions are primarily addressed in [Hrishikesh2002], a seminal ISCA 2002 paper, connecting VLSI design with processor architecture (or microarchitecture design) using the method of logical effort.

[Harris2001] also provides secondary information supporting my answers.

1) What is meant by deeper pipelines?

From [Hrishikesh2002] and [Harris2001], deeper pipelines refer to VLSI circuits and systems, including VLSI implementations of processor architectures (or microarchitecture designs), with more pipeline stages (e.g., 20-stage pipelined ARM processor versus a 5-stage pipelined ARM processor).

You got this right.

2) Why do they allow for higher clock frequencies

You got this right, too.

When you split a given circuit into more pipelined stages, such that each stage has approximately the same delay/latency for the combination circuits between the pipelined stages, the combination circuits between the registers (usually implemented with flip-flops) for each pipeline stage will be smaller and consequently have smaller delays/latency.

It is the worst-case delay/latency for the pipelined stages that determine clock frequency. The clock frequency is the inverse of the worst-case delay/latency.

Hence, smaller delays lead to faster clock frequencies.

3) Given a deeper pipeline, why does it follow that they use more power?

The first reasoning is correct, and the second reasoning is quite close.

[Weste2011, Equation 5.13, pp. 185] refers to the total power consumption of a digital IC or VLSI circuit/system being the sum of its total dynamic power consumption and total static power consumption.

[Weste2011, Equation 5.10, pp. 184] states that the dynamic power consumption due to the switching load is proportional to the clock frequency. Hence, the higher the clock frequency, the higher the dynamic power consumption due to the switching load/activity.

As aforementioned, deeper pipelines would have more aligned registers to connect the outputs of the combinational circuit in the previous pipelined stage to the inputs of the combinational circuit in the next/current pipelined stage. Hence, more registers (or specifically, their flip-flop implementations) mean more dynamic and static power consumption.

If you think the "unrolled" sequential circuits in [Harris2001] and [Patterson2021] has an unfavorable aspect ratio for the VLSI design constraints for the area/volume of your I.C., you can "roll it up" using the Huffman model [Navabi2011, Section 7.1.2.1 on "Sequential Circuit Huffman Model", Figure 7.1, pp. 214] [Miczo2003, Section 2.5 on "Sequential Circuit Behavior", on pp. 39-40, including Figure 2.2]. [Abramovici1990, Section 2.2.2 on "State Tables and Flow Tables", pp. 13-14, including Figure 2.5 on pp. 14] refers to it as the canonical model of synchronous sequential circuits. [Jha2003, Section 11.2 on "Scan design", pp. 562, Figure 11.1] refers to it as the general model of (synchronous) sequential circuits.

Essentially, in terms of visualizing the schematic, "rolling it up" would vertically stack the pipeline stages on the combinational circuit side and also on the sequential element side (a much longer column of registers, typically implemented with flip-flops). The "unrolled" version in [Harris2001] and [Patterson2021] spreads the combinational circuits across visible pipeline stages (demarcated by columns of registers/flip-flops between pipelined stages) and consequently reduces the height of each column of registers/flip-flops (split the super tall/long column into multiple equivalent columns of registers/flip-flops). The total number of logic gates/cells in the combinational circuits (forming the data path) and the total number of registers/flip-flops is the same for either representation. The key difference is the aspect ratio of your final VLSI implementation, in terms of layout area.

You have to pick the unrolled or rolled up version based on your VLSI design objectives and constraints, in terms of performance/delay, energy efficiency (or average/total power consumption), area or volume (for 3-D I.C.s), and any other metric of interest.

E.g., dumping all the registers/flip-flops together can make it easier to design the clock network/tree, but cause problems with electromagnetic interference (such as capacitive crosstalk or noise). Do you really want to lump all the registers/flip-flops together, or spread it out fairly evenly across the layout?

An aside: The terms unroll and unrolling are analogous to loop unrolling in compiler design [Muchnick1997, Section 17.4.3 on "Loop Unrolling," pp. 559-562] [Aho2007, Section 10.4.5 on "Global Scheduling Algorithms," pp. 735] [Cooper2012, Section 8.5.2 on "Loop Unrolling," pp. 441-444] and high-level synthesis [Elliott1999, Section 6.7 on "Loop Unrolling," pp. 98-102] [Coussy2008, Section 3.3.3.1 on "Loop Unrolling," pp. 40-41] [Bailey2007, Section 3.8.2.1 on "SPARK Parallelizing High-Level Synthesis (PHLS)," pp. 61-62]

4) Why does a less deep pipeline not have the same performance issues if there are dependencies between operations in the pipeline?

[Harris2001] is a seminal textbook to show readers/students how to pipeline digital/logic circuits for faster performance (in terms of delay/latency). [Hinton2001] provides an example of a pipelined adder for faster performance in the design of the Pentium IV processor.

Together, [Harris2001] and [Hinton2001] show us how to deepen the pipeline stages in processor architectures [Patterson2021]. Resource contention leads to hazards that arise from the dependencies between operations of digital circuits (such as instructions of processor architectures), from the classic control hazards in (general-purpose) processor architectures to the data hazards associated with overwriting data in registers shared/used by consecutive instructions/operations". E.g., the write after read (WAR) data hazard (for general-purpose processor architectures) reflects how the output of the subsequent write operation can overwrite the original data value of a register associated with the initial read operation, when the write operation is completed earlier than the read operation. [Patterson2021] describes using pipeline bubbles, operand forwarding, and out-of-order execution to mitigate/avoid these data hazards.

So, you partially got this correct.

When you integrate these concepts together, pipeline depth does matter, since it affects the performance/speed/delay/latency [Hrishikesh2002] of your digital/logic circuits.

Since performance/speed/delay/latency is a key metric for pricing products based on digital circuits or electronic systems, including processors, digital IC designers (or VLSI designers) do have to care about pipeline depth. If the product team you work for cannot sell its slower products, the product team has to pivot to work on a new line of products or get laid off.

Likewise, if an academic research team cannot design faster digital circuits and VLSI/electronic systems, it cannot publish enough research papers to help students graduate (important for Ph.D. students and Master's students working on their Master's thesis, or even good undergraduates working on their honors research projects). For researchers in corporate research labs, if they cannot design faster circuits and VLSI systems, they cannot publish enough high-quality research papers. If they do not publish enough high-quality research papers, they can get laid off, too.

References

@book{Weste2011,
    Address = {Boston, {MA}},
    Author = {Neil H. E. Weste and David Money Harris},
    Edition = {Fourth},
    Publisher = {Pearson Education},
    Title = {{CMOS VLSI} Design: A Circuits and Systems Perspective},
    Year = {2011}}

@book{Harris2001,
    Address = {San Francisco, {CA}},
    Author = {David Harris},
    Publisher = {Morgan Kaufmann},
    Title = {Skew-Tolerant Circuit Design},
    Year = {2001}}

@inproceedings{Hrishikesh2002,
    Address = {Anchorage, {AK}},
    Author = {M. S. Hrishikesh and Doug Burger and Norman P. Jouppi and Stephen W. Keckler and Keith I. Farkas and Premkishore Shivakumar},
    Booktitle = {Proceedings of the $29^{th}$\ Annual International Symposium on Computer Architecture ({ISCA '02})},
    Doi = {https://dx.doi.org/10.1145/545214.545218},
    Month = {May 25--29},
    Organization = {{ACM} Special Interest Group on Computer Architecture and {IEEE} Computer Society ({TCCA})},
    Pages = {14--24},
    Publisher = {{IEEE} Press},
    Title = {The Optimal Logic Depth Per Pipeline Stage is 6 to 8 {FO4} Inverter Delays},
    Url = {https://dx.doi.org/10.1145/545214.545218},
    Year = {2002}}

@book{Navabi2011,
    Address = {New York, {NY}},
    Author = {Zainalabedin Navabi},
    Doi = {https://dx.doi.org/10.1007/978-1-4419-7548-5},
    Publisher = {Springer Science+Business Media, {LLC}},
    Title = {Digital System Test and Testable Design: Using {HDL} Models and Architectures},
    Url = {https://dx.doi.org/10.1007/978-1-4419-7548-5},
    Year = {2011}}

@book{Patterson2021,
    Address = {Cambridge, {MA}},
    Author = {David A. Patterson and John L. Hennessy},
    Edition = {Second {RISC-V}},
    Publisher = {Morgan Kaufmann},
    Series = {The Morgan Kaufmann Series in Computer Architecture and Design},
    Title = {Computer Organization and Design: The Hardware/Software Interface},
    Year = {2021}}

@book{Abramovici1990,
    Address = {Rockville, {MD}},
    Author = {Miron Abramovici and Melvin A. Breuer and Arthur D. Friedman},
    Publisher = {Computer Science Press},
    Title = {Digital Systems Testing and Testable Design},
    Year = {1990},
    Annote = {Breuer and Friedman, ``Diagnosis {\rm \&}\ Reliable Design of Digital Systems.'' See \cite{Breuer1976} for the ``earlier edition'' of this book.}}

@book{Jha2003,
    Address = {Cambridge, {U.K.}},
    Author = {Niraj Jha and Sandeep Gupta},
    Publisher = {Cambridge University Press},
    Title = {Testing of Digital Systems},
    Year = {2003}}

@book{Miczo2003,
    Address = {Hoboken, {NJ}},
    Author = {Alexander Miczo},
    Edition = {Second},
    Publisher = {John Wiley {\rm \&}\ Sons},
    Title = {Digital Logic Testing and Simulation},
    Year = {2003}}

@article{Hinton2001,
    Address = {Hillsboro, {OR}},
    Author = {Glenn Hinton and Dave Sager and Mike Upton and Darrell Boggs and Doug Carmean and Alan Kyker and Patrice Roussel},
    Journal = {Intel\textsuperscript\textregistered\ Technology Journal},
    Month = {Quarter 1},
    Pages = {1--12},
    Publisher = {{Intel} Press},
    Title = {The Microarchitecture of the Pentium\textsuperscript\textregistered\ 4 Processor},
    Year = {2001}}

@book{Elliott1999,
    Address = {Norwell, {MA}},
    Author = {John P. Elliott},
    Doi = {https://dx.doi.org/10.1007/978-1-4615-5059-4},
    Publisher = {Kluwer Academic Publishers},
    Title = {Understanding Behavioral Synthesis: A Practical Guide to {High-Level} Design},
    Url = {https://dx.doi.org/10.1007/978-1-4615-5059-4},
    Year = {1999}}

@book{Coussy2008,
    Address = {Dordrecht, {The Netherlands}},
    Editor = {Philippe Coussy and Adam Morawiec},
    Doi = {https://dx.doi.org/10.1007/978-1-4020-8588-8},
    Publisher = {Springer Science+Business Media {B.V.}},
    Title = {High-Level Synthesis: From Algorithm to Digital Circuit},
    Url = {https://dx.doi.org/10.1007/978-1-4020-8588-8},
    Year = {2008}}

@book{Bailey2007,
    Address = {San Francisco, {CA}},
    Author = {Brian Bailey and Grant Martin and Andrew Piziali},
    Publisher = {Morgan Kaufmann},
    Series = {The Morgan Kaufmann Series in Systems on Silicon},
    Title = {{ESL} Design and Verification: A Prescription for Electronic System-Level Methodology},
    Year = {2007},
    Annote = {The Morgan Kaufmann Series in Systems on Silicon
        Series editor: Wayne Wolf}}

@book{Muchnick1997,
    Address = {San Diego, {CA}},
    Author = {Steven S. Muchnick},
    Publisher = {Academic Press},
    Title = {Advanced Compiler Design and Implementation},
    Year = {1997}}

@book{Aho2007,
    Address = {Boston, {MA}},
    Author = {Alfred V. Aho and Monica S. Lam and Ravi Sethi and Jeffrey D. Ullman},
    Edition = {Second},
    Publisher = {Pearson Education},
    Title = {Compilers: Principles, Techniques, {\rm \&}\ Tools},
    Year = {2007}}

@book{Cooper2012,
    Address = {Burlington, {MA}},
    Author = {Keith D. Cooper and Linda Torczon},
    Edition = {Second},
    Publisher = {Morgan Kaufmann},
    Title = {Engineering a Compiler},
    Year = {2012}}
```