How to communicate faster than the system clock

Question

I was reading about the new (ish) Thunderbolt 3 today, and was very impressed by the specced speed of 40Gbps. Then I looked at Intel's latest i9 processor speed... about 4.2GHz max. How can a system communicate almost 10 times faster than the clock driving it? Even parallel communication would still need 10 channels to get 40Gbps, and from what I can tell USB-C only has 4 or so.

Having read the responses, it seems like there are two parts to this: how the CPU/system communicates at such high speed, and how that high data rate (which it seems is accomplished using many parallel lines) gets through the bottleneck of the thunderbolt cable itself, which only has a small number of parallel channels.

Answer 1

I guess the confusion is you assume you can only send one bit per clock cycle. There are lots of ways a communication scheme can essentially encode more than one bit per symbol. A symbol is abstract idea as the atom of transfer in a communication system.

It's really too big of a topic to cover in an answer to this question in any depth, but imagine you weren't constrained to binary values and instead could send one of 1024 voltages as a symbol. In effect that would be 10 bits of information per symbol, and you would get 10x the "clock speed" in bandwidth. That's how the old NTSC video encodes data for example.

Another way of getting that kind of bandwidth is using buffering and then delegating the transport to specialized transmitters and receivers, sometimes called SERDES (for serializer / deserializer) blocks. You can't sustain the throughput that these are capable of without a source or sink capable of keeping up, but you can reduce the latency of the transfer of blocks of information between computing nodes using something like that. Look up Phase-Locked Loops (aka PLLs), as FPGAs and ASICs can use these to derive faster clocks from a basis clock to do this sort of thing.

Another way is simply having lots of parallel channels to transmit the data on. Think of the old parallel ports on PCs, in a single clock you transfer a whole bunch of bits, each bit on its own dedicated wire. USB-C and its kin have a lot more data pins than one RX and one TX for example.

Bandwidth is an aggregate property involving the net effect of all kinds of techniques like these, because many can be used together even.

Answer 2

The other answers have focused more on the Thunderbolt side of things, but let’s look back at the statement

Intel's latest i9 processor speed... about 4.2GHz max

4.2 GHz is the system clock, which (in a very, very simplistic way) is comparable to the number of instructions per second, per core (it’s really a lot more complex than that, as not all instructions take the same time to execute, there are wait times, etc.).

But on each cycle, the CPU will process data (to/from registers, caches, RAM and possibly I/O, from fastest to slowest). In the meantime, other peripherals can also read from/write to RAM without the CPU being involved (that’s called DMA).

The main bottleneck is then often RAM. It needs to be fast enough to feed the CPU as needed (for instructions to run and data to process), do DMA, and is in some cases shared with a GPU or acting as a frame buffer for video (there is then some component that is reading the RAM that acts as a frame buffer to send it to a video output, on each frame — for a Full HD 1920 x 1080 resolution at 60 Hz with 24-bit colour, that's nearly 3 Gbits/s, for 4K@60 fps, 4 times that).

RAM uses wide buses, usually 32 or 64-bits wide. There may be several separate channels. The fastest RAM currently seems to be DDR4-3200, which allows 3200 million 64-bit transfers per second. That's 25600 Mbytes/s or 204800 Mbits/s (over 200 Gbits/s), per channel.

An i9-9980XE CPU can have 4 memory channels. That means RAM could support over 800 Gbits/s, so 40 Gbps is just peanuts compared to that.

The impressive part about Thunderbolt is getting that speed over longer distances (not a few cm on a motherboard), over a relatively simple cable (not hundreds of connectors, as required to support multiple 64-bit RAM buses).

Answer 3

There is no direct correlation between processor speed and peripheral speed.

It is not 10 ×. Thunderbolt 3 - \$\frac {40 Gbps} {8 bits/byte} = 5 Gbyte/s\$. This rate does not even seem unrealistic for a 4.2GHz 64-bit processor.

But that is not what we are dealing with here. We have a peripheral with serial communication + graphic card. Four times data + twice video bandwidth of existing capabilities. As the link says, desktop performance from a laptop. One port to link them all and in the darkness, bind them.

From Thunderbolt 3 – The USB-C That Does It All

Users have long wanted desktop-level performance from a mobile computer. Thunderbolt was developed to simultaneously support the fastest data and most video bandwidth available on a single cable, while also supplying power. Then recently the USB group introduced the USB-C connector, which is small, reversible, fast, supplies power, and allows other I/O in addition to USB to run on it, maximizing its potential. So in the biggest advancement since its inception, Thunderbolt 3 brings Thunderbolt to USB-C at 40Gbps, fulfilling its promise, creating one compact port that does it all.

---

There are multiple clocks within a CPU; peripherals can run much faster than the listed CPU speed, either by running off a faster clock or by implementing parallel communication.

No and no and no.

There are multiple clocks in a computer. Inside a CPU there is one clock. Peripherals can either derive their clock rate from the system clock (slower) or use a crystal to make their own clock.

Parallel communications have gone the way of the dodo, obsolete. Parallel communications were limited to short distances. USB, I2C, I2S, CAN, etc. are all serial protocols.

Your 4.2GHz processor does not communicate at 4.2GHz. That's the clock rate, a better indication is MIPs. And that is program instructions, not communicating externally.

You cannot equate a 64-bit processor running at 4.2GHz with a peripheral running serially at 20GHz. The 20GHz clock is not derived from the 4.2GHz. At 20GHz, the frequency is more analog than digital.

Now bit-banging, properly designed, the 4.2GHz 64-bit processor could probably do 20GHz serially (5 Gbyte/s), but that's not it's purpose.

Answer 4

There are many techniques to encode or decode a large amount of information into one "communication bucket/wagon" (which are called symbols). The R-2R-type Digital-to-analog converter (illustrated below) is one of the simplest yet effective methods of transmitting e.g. 16x faster than the communication clock - provided that the receiver's Analog-to-digital converter converts within a clock period (main limitation: noise).

Nowadays, analog communication not only use amplitude, but phase information to stuff even more bits in a communication clock cycle, as illustrated in This topic (Disclaimer: I have one answer there, but it's the only topic I can remember of which discusses this). And this is only a mere example of a plethora of techniques that become more and more complex with every year that goes by.

For digital communication, it's mostly down to: 1) number of parallel lines (1 for serial communication, USB is a serial link even though it has two data lines, they are always opposite each other in value to increase robustness to common mode noise by difference), and 2) compression. When you send 100MB worth of files to a 10MB-limit mailbox by compressing it, you're effectively transmitting in one clock cycle more than the physical uncompressed datarate limit. Just as there are many modulation techniques (seen above), there are many compression techniques. Note that there are dedicated compression/decompression chips available to take full advantage of compression as a means to increase transmission speed. Note that I am not saying that's what is being done for video transfers, since compression/decompression adds latency. It's however one method used for satellite links though, for example.

Finally, datarate limits are really limits for a given error rate. If you add to your communication protocol some encoding which detects errors and corrects them automatically (using for example state machines on both ends), you can increase the datarate past the stated limit for the same error rate.

That's how data gets in and out of a computer faster than the CPU clock, or how "one man/woman can move more or less buckets from your house to another" (amplitude and/or phase modulation, compression, encoding...).

Now, for the "how to get the buckets from your door to wherever you want it in your house", that's down to the width of your databus: 64bits computers took over from 32 bits computers for this reason, handle twice more data for every clock cycle. That's the number of buckets that can be grabbed at the same time by the same person in your house (assume you live on top of radioactive waste). I think processing cores could have an impact on pure transfers if the clocks of the cores are phased out to move data quicker on the data bus but generally they share not only the same bus but the same clock (the maximum clock rate is certainly set to the maximum propagation time of the data along the bus). This is obviously a simplistic view, many factors come into play - but it has the benefit of adding more reasons why it IS possible.

Answer 5

First of all Thunderbolt 3 is not really 40 Gbit/s for data. It is maximum 32.4 Gbit/s for data, everything else is DP, which is separate and BTW can limit what amount of data is available to you. Furthermore, that data is 8b/10b encoded, so the usable bandwidth drops to 25.92 Gbit/s (32.4 / 10 * 8). Now, we know that PCIe 3.0 four lanes are actually 3.938 GB/s full duplex, or if we will x8 it: 31.504 Gbit/s. So it is not even all of available PCIe bandwidth!

Second of all 4.2 GHz means nothing. Internally (in VIS framework, for example, for debug) Intel already uses PICOsecond precision clocks, okay, and it was back in 7th generation. And we do not know what it is in BigCore (which is what you call CPU, Intel calls it BigCore and it is just a dumb extension of the main thing, the chipset). So the Minix OS inside the CPU/chipset is certainly not 4.2 GHz but at least x100 of that and Intel can utilise it if it needs it, which in most cases it does not, as it is hardware accelerated.