2
\$\begingroup\$

Typical CPU instruction set: the CPU has several functional units, and when each instruction is read, some bits specify which functional unit is to be activated, while others specify the details of the operation.

Today's CPUs tend to spend some of their large transistor budgets on out of order execution, where they have a bunch of instructions in flight at once, trying to keep multiple functional units busy all while creating the illusion that it's still a serial instruction stream for backward compatibility. For the purposes of this question, to keep it simple, say we are talking about 90s technology with no more than a few million transistors.

VLIW: let's save time and transistors on decoding, and keep all the functional units busy, by having an instruction word that has a group of bits for each functional unit, specifying work for them all at the same time.

One problem with VLIW is it exposes more hardware details, making it difficult to keep backward compatibility with subsequent CPU generations without losing the advantages. Let's say we get around that problem with byte code and a JIT compiler.

Another problem is that not all workloads want to use all the functional units, and that's what I'm looking at now.

Let's say we're designing a CPU for an early 90s workstation, in the era of the 486 and i860. Roughly speaking, there will be two kinds of workloads, number crunching (e.g. CAD, simulations and of course 3D games) that uses the floating point units, and other stuff (e.g. word processing, compiling) that doesn't; this roughly corresponds to SPECfp vs SPECint.

For number crunching workloads, we could design a straight VLIW, maybe with a 64-bit instruction word that specifies integer and floating point operations, control flow etc all at once, and that will give great performance. But for integer workloads it will have poor code density because the bits specifying floating point operations will always be saying 'no FP ops this clock cycle'.

Would it make sense for the CPU to have a mode bit to switch between the two kinds of workloads? In FP mode, it works like the above, but in integer mode, maybe each 64-bit word just supplies a pair of 32-bit integer instructions? Would that give most of the simplicity and performance advantages of VLIW without the potential code density disadvantage?

\$\endgroup\$
  • \$\begingroup\$ It kind of depends on when the switching happens. If you use coarse-grained switching, like how desktop CPUs switches privilege mode, then you won't be able to mix integer and FP instructions into one VLIW word. If you switch at each instruction, you are essentially uniting integer and FP instructions into one instruction space and possibly both would need to share the register file, which in practice makes it hard to scale to high frequency. (e.g. TI's C66 is such a united architecture and usually runs at 1.2GHz @ 28nm) \$\endgroup\$ – user3528438 Sep 5 '17 at 16:20
  • \$\begingroup\$ The only way to find out if this makes sense is to design it and see if anybody buys it. \$\endgroup\$ – The Photon Sep 5 '17 at 16:20
  • 2
    \$\begingroup\$ The performance advantages of VLIW were rather debatable from the beginning, and the early 90s is exactly when that was starting to become apparent -- as the gap between core processor speed and memory speed widened, the benefits of encoding compactness were becoming very apparent. ARM went the opposite direction: adding a mode to their processors that allowed smaller, simpler instructions. And the compactness of x86 code is one of the key reasons it managed to keep up performance-wise with the RISC architectures and stay dominant through this time. \$\endgroup\$ – Jules Sep 5 '17 at 16:25
  • \$\begingroup\$ I therefore don't think any kind of hybrid would help at all. There was ongoing research at the time into dataflow architectures as another way of increasing functional unit utilisation, but that turned out to be another hard problem. But some kind of limited dataflow could have been worked out and might have made sense back then if anyone had really thought about how to bring dataflow inside a standard von neumann architecture, rather than completely replacing it as most researchers were thinking at the time. \$\endgroup\$ – Jules Sep 5 '17 at 16:41
  • 1
    \$\begingroup\$ @user3528438 I'm talking about coarse-grained switching, not between integer only and floating point only, but between integer only and integer plus floating point (since number crunching workloads still also need integer instructions). \$\endgroup\$ – rwallace Sep 5 '17 at 16:51
7
\$\begingroup\$

Go read the book, "Bulldog: A Compiler for VLIW Architectures," by John R. Ellis. It is absolutely superb.

As you might suspect, one of the huge problems with VLIW is about the compilers themselves. How do you get compiler authors (I've been one of them for a short period of my life) to decide to add in all the needed stuff to make VLIW work out?

You admit that "one problem with VLIW is it exposes more hardware details," but you failed to take that in the direction you needed to go. VLIW exposes the hardware details to the compiler writers!! And they are the ones you need to support your CPU!

Frankly, that's going to be very hard. It's already been like pulling teeth to get far more "run of the mill" optimizations into their compilers. Some have happened, such as partial template specialization for C++ (very much needed to reduce code "bloom".) But even then, it took forever. And there are techniques I learned about in the 1970's that still haven't found their way into compilers today, or are just barely finding purchase.

For example, John's Bulldog compiler performed transformations that would move code across edges (conditional boundaries generated by an "if" statement.) So code that followed an "if" would be moved above the "if" and executed speculatively, tossing the results if the "if" statement went a different direction. The coder had the ability to provide "hints" to the compiler so that the compiler could gain some added information about which condition was more likely to occur. The compiler included features to deal with DRAM banks so that data could be placed in such a way to minimize the fetch times (cache was more expensive then) based on order of access in loop structures.

It's really a very good book to read and I think you'd enjoy it.

But the key here is getting compiler writers, those in control of compilers that people actually use a lot, to implement all this.


To give you a flavor of just how all this played out, note that the MIPS R2000 RISC processor could beat the daylights out of the Intel x86 core in 1986. (I was there, working on R2000 plug in boards for the IBM PC and writing operating system code for all this and I spent a while at MIPS getting 1:1 training directly from John Hennessy.) And they used FAB technology that was ages old to do it, too. Intel and Motorola then had the best FABs in the world (and would not share them, of course) and MIPS had to use hand-me-down FAB technology with larger feature sizes and a tenth or less of the transistor count. And they still slaughtered Intel's x86 by a mile.

So what did Intel do? Well, of course they started up their own RISC projects. But what really changed the game was all the things you already talked about in your question. (I was working at Intel on their chipsets while this was happening.)

Rather than throw away the x86, they leveraged their huge FAB advantage and instead added parallel instruction decoders so they could decode up to three instructions per clock. They added a RISC engine (yup, just created one and popped it into a corner of their die) and a "re-order buffer" (ROB) to hold RISC instructions that were decoded out of the CISC x86 instructions by the decoder. They added a registration station so that they could run functional units in parallel. They added a retire-unit so that while they executed the instructions out of order, they had a means to put that back into order in terms of how it appeared to the outside. And the retire unit could retire three RISC instructions per clock, too. They added branch prediction, caches (several levels), and a lot more.

In short, they THREW transistors at the problem. And they had them to spare, too.

In the end, they were able to hold out long enough that MIPS and the Motorola 88k and the DEC Alpha and their own internal RISC projects became pointless, again. Technology made the early "bare metal" advantages disappear by the parallel growth in FAB technology and going from million transistor dice to billion transistor dice. It didn't take that long, actually -- 10 years maybe? -- before those initiatives became far less relevant again.

And so here we are.

If you could get significant compiler development effort to support VLIW, you might get some motion with the then-exposed-to-view functional units. But really? There really today isn't that much of an advantage that cannot be addressed by just throwing some corner of the die at it, now. There was a day when that meant something. But today the problem isn't getting the most out of the hardware you have, but what to do with all those transistors they have available now. For gosh sake, they just turn them into cache as the best thing they can do with some?

And the compiler writers are going to be similarly "not interested," as well.

In my opinion the world has changed here. They have so many transistors that they have little idea what to do with. They don't have the problem of not enough transistors that they need to make better use of, that they once had.

\$\endgroup\$
  • 1
    \$\begingroup\$ "They have so many transistors that they have little idea what to do with." - It's worth adding that this mostly applies to the desktop and server markets, where there's money and power for 1-billion transistor CPUs. In the embedded market (to some extent including phones), size, cost, and power are still very relevant. This is reflected in Intels market position in that world: virtually non-existent, while RISC-likes ARM, MIPS and PPC thrive there. \$\endgroup\$ – marcelm Sep 5 '17 at 18:32
  • \$\begingroup\$ @marcelm True enough. But that's the area where VLIW was targeted, so it's already "cross-purposes" when we are talking at that level. \$\endgroup\$ – jonk Sep 5 '17 at 19:19
  • \$\begingroup\$ Sure, I just thought it would add a little extra context to that part of the answer :) \$\endgroup\$ – marcelm Sep 5 '17 at 21:16
  • 1
    \$\begingroup\$ Knuth had a well-known quote about VLIW: "The Itanium approach...was supposed to be so terrific—until it turned out that the wished-for compilers were basically impossible to write." \$\endgroup\$ – Mark Sep 6 '17 at 0:59
5
\$\begingroup\$

Don't underestimate the importance of cache...

Cache memory is much faster and much more expensive per byte compared to SDRAM. When a branch prediction fails, you'll get a pipeline stall, but if the instruction being branched to is in cache, it's a small loss. If it is in SDRAM though... an it wasn't prefetched... it's more like a catastrophic failure.

So a compact instruction set (like mentioned by Jules) means your cache holds much more code. In practice this is a lot more important than spiffy VLIW.

Let's say we're designing a CPU for an early 90s workstation, in the era of the 486 and i860.

This is basically a Cortex-M4. Clock frequency is not that fast, thus pipeline can be short, making branch prediction less of a hassle. But you still need compact code to reap the benefits of that 16kB stick of SRAM cache you bought at ludicrously high cost to put in your PC...

Roughly speaking, there will be two kinds of workloads, number crunching (e.g. CAD, simulations and of course 3D games) that uses the floating point units, and other stuff (e.g. word processing, compiling) that doesn't; this roughly corresponds to SPECfp vs SPECint.

You forgot the other workload everyone forgets about: control flow.

This happens typically in interpreters, or any other code where most of the stuff consists of tests, conditional jumps, grab a function pointer and go to it, etc. Even printf() is a crude interpreter.

Think about a web browser parsing html, or handling xml, it's all control flow. Or a server running php.

Also Excel: there is an interpreter which processes your formulas. Updating the spreadsheet is like 1% doing actual FPU, and the rest is if(), case() and jumps.

The big problem with this is that you'll have a few main loops in your code, with tons of conditionals. Even a smart branch predictor won't handle these, because the last time it was in this bit of code, it was parsing a different character, or executing a different bytecode. Even if the predictor remembers which branch was taken last time, in this scenario it is useless. The pipeline will stall a lot!

In this scenario, and if there are enough threads, reallocating silicon area from complex branch prediction/superscalar/out of order and other fancy features towards more dumb cores with lots of fast cache makes sense.

If you're encoding x264 video, things are completely different: you want tons of memory bandwidth, SSE instructions to crunch several pixels at a time, prefetch, etc.

For number crunching workloads, we could design a straight VLIW, maybe with a 64-bit instruction word that specifies integer and floating point operations, control flow etc all at once, and that will give great performance. But for integer workloads it will have poor code density because the bits specifying floating point operations will always be saying 'no FP ops this clock cycle'.

Yeah.

Crunching means loops, and this means "apply one instruction to several values at high throughput." SSE, MMX, ARM NEON are very good answers to this. VLIW, not so much.

Bigger crunching these days means a GPU.

Would it make sense for the CPU to have a mode bit to switch between the two kinds of workloads?

No, there was an arch that had this and failed because of it, I don't remember the name though.

The evil in this is that when you call a function from your code, how does it know in which mode the cpu is? Maybe it'll trigger a mode switch (slow)... It creates all sorts of dependencies.

Also if int and fp registers are shared, the hardware that manages superscalar and instruction dependency now has 2x more work because each register can receive values from more different sources...

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.