One of the most common optimizations used in modern processors is to keep the silicon as busy as possible. Cache units access memory for the processor so other circuitry isn't tied up for the dozens of clock cycles it takes to access memory. Branch predictors speculatively schedule instructions so that the processor does not have to wait for a branch to fully resolve to pipeline instructions after a branch. Instructions are executed out of order so that instructions that follow it aren't unnecessarily delayed. Anything that can be parallelized in a processor, is.

Except apparently the ALU.

As far as I'm aware, ALUs are integrated circuits that direct operations to one of several individual operation circuits based on the operation specified by an operation field. ALUs are combinatorial: if a processor sets the ALU to perform a operation, such as multiplication, then while the ALU runs that specific computation, other functionality within the ALU, such as addition, is inaccessible while the multiplication still runs. It would appear to me that a processor that had separate logic circuitry for each operation rather than an ALU could run both operations in parallel, yielding performance gains for code that heavily uses multiple arithmetic operations.

So why isn't this done?

(Or at least, why isn't it common or noteworthy enough to be written about in literature about computer architecture?)

  • 4
    \$\begingroup\$ Actually the kinds of things you are proposing are done. But such optimizations are more readily applied to special purpose computational engines like DSP's and GPU's (and of course custom constructs in FPGA's and ASIC's). Trying to do these fine grained parallelisms while running a program written in a sequential language requires all sorts of local reshuffling tricks - the challenge is not in building the datapaths, but the in code generator and execution logic algorithms to permit automatically making use of them. \$\endgroup\$ Nov 29 '17 at 20:39
  • 5
    \$\begingroup\$ One reason is that an ALU is just a small thing on the scale of a modern CPU. When the one ALU is a bottleneck, just include two. Or more. The real performance bottleneck on modern desktop CPUs is the memory system, so instead of ALUs the chip developers include (more, faster) cache. The main reason modern chips contain more than one CPU (or a multi-issue CPU, or hyperthreading) is to make more effctive use of the memory. \$\endgroup\$ Nov 29 '17 at 20:47
  • \$\begingroup\$ ALUs are not serial (bit-serial) and are carefully designed so that their longest path is just under a clock cycle. There are CPUs with more than one ALU; scheduling for them is hard enough even when each has a full set of operations. (and multiply is usually offloaded to a dedicated unit where performance matters). \$\endgroup\$ Nov 29 '17 at 21:04
  • 2
    \$\begingroup\$ If you need a lot of mathematical operations done, why wouldn't you use SIMD? Cpu architects do figure out at the assembly level what frequent math operations happen in common code and then create dedicated instructions to do so as Spehro points out. \$\endgroup\$
    – horta
    Nov 29 '17 at 22:10

If addition is inaccessible while multiplication is still running it is because multiplication requires also some addition operations (together with some shifting).

Otherwise the ALU needs to have more than one adder (which is not unrealistic at all).

BTW: If the ALU can't do more than one arithmetic operation at the same time I wouldn't call that "serial" (because it reminds too much of serial adder; which performs adding operation bit by bit).
If the CPU can't do more than one instruction at the same time (in one cycle) it is just called "not super-scalar".


There is the Dark Silicon problem: Every computation generates some heat. Having each and every part of the ALU doing computation at the same time can generate enough heat to melt down the chip. Giving the different parts of the ALU some rest every now and then helps to spread the heat and keep the peak temperature low enough for save operation. (This applies mostly to very fast and highly dense CPUs such as the CPU in your PC. Microcontrollers aren't usually affected by this).

For not so fast architectures this is usally not a big deal and specialized architectures that try to keep all aspects of the ALU busy do exist. Without going to much into detail the DSP C6x architecture from TI for example allows some crazy things. It has eight specialized ALU pipelines which allow roughly up to two dozen instructions active at the same time. While this sounds great from an performance point of view it has a serious drawback: Writing code that keeps all pipelines busy is very hard. Debugging the code is even harder. You get great ALU performance per watt out of such an architecture but the development costs explode.


First of all, you should be aware that you're basically wrong: while there certainly are limits, a modern CPU can execute quite a few instructions simultaneously.

Let's use AMD's new Zen architecture as an example. Its current incarnation supports dispatching (i.e. starting execution of) up to 12 micro-ops per clock cycle (6 integer, 6 floating point--though there's some disagreement about that--some sources say only 4 floating point).

Although that clearly could be increased, doing so would be non-trivial. Just for one obvious point, it's not just a matter of being able to execute more micro-ops per clock.

For integer operations, you need to support 128 bits of inputs for many of the simple ops (add, subtract, and, xor, not), but you have a few operations that need even more (e.g., integer division takes one normal size and one double-sized input, so it takes 192 bits of input operands).

Worse, to get a decent chance of actually executing more ops per clock, you need a larger pool of decoded ops available to dispatch. You also probably need to have even more physical registers (AMD's current design uses 168) to support more register renaming and give yourself a chance of finding more ops to actually execute in a given clock cycle.

So, your design quickly balloons. Right now, AMD apparently needs around two dozen 168-input MUXes to retrieve inputs from the registers into the execution units (of widths varying from 64 bits up to 256 bits), and another dozen (or so) demuxers to store the results.

If you wanted to double the possible execution rate, you'd obviously need to increase the number of MUXes from two dozen to four dozen. Worse, however, to give yourself a chance of actually executing more instructions, you'd want to increase your physical register bank from 168 to somewhere around 336 registers, so not only do you need twice as many MUXes, but each one becomes substantially larger as well. Likewise, to store your results, you'd need to bump it up from one to two dozen demuxes--and, again, each one would need to get larger to support more physical registers.

Then you have the fact that you can currently only decode 4 instructions per clock. If you want to execute 24 ops per clock, you probably just about need to be able to decode around 8 instructions per clock--being able to execute 24 micro-ops per clock won't gain much if you can't decode that many.

Unfortunately, x86 instructions are not easy to decode--so doubling the number of decoders is a pretty expensive proposition as well.

Bottom line: a decent CPU design depends on balancing resources throughout the design. If you want to be able to actually accomplish anything by being able to dispatch more ops per clock, you're going to need to make changes throughout nearly the entire design.

All of that, of course, could be done--but it starts to use up a lot of silicon very quickly. At the present time, it would appear that you gain more by Using that silicon in other ways (more cores, bigger cache, more memory controllers, more PCIe lanes, etc.)


In at least one case it is very commonly done- DSP functionality usually has a MAC instruction (multiply-accumulate) because that is a very frequently performed operation in digital signal processing.

I don't think use of different parts of the ALU independently would be"free" though, it might be cheaper in terms of silicon area to have a second adder rather than trying to use one for two purposes- there might have to be additional bus entry points or whatever, and because the gates required for an adder only scale with the number of bits n. Parallel multipliers scale as n^2, so they are going to be expensive to implement, especially for n large.


The Multiflow Corporation sold machines with 128-bit and with 256-bit instructions. Lots of activities were done in parallel. The key was the compiler.

Intel bought Multiflow decades ago.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.