How is quality/damage control done in large circuits?

There is apparently more than 28 billion transistors in the recently released Nvidia GeForce RTX 3080.

First, when you manufacture such a circuit there must be a decent chance of botching some of these. How do the plants make sure that they all work?

Second, after prolonged usage surely at least one of these transistors must fail. Does the circuit somehow recognize this and redirect the workflow?

• Look up wafer scale integration. – copper.hat Oct 13 '20 at 15:41
• Design defects are a wholly different thing from manufacturing defects. For the latter there is testing, and then the practice of selling at a lower price/performance tier chips where only some of duplicated paths test as good. Overall this question is insufficiently focused to fit on this site, which is reserved only for questions which can have a specific concise answer. – Chris Stratton Oct 13 '20 at 19:11
• @ChrisStratton the question doesn't mention design defects. – lfds Oct 14 '20 at 7:08

We don't "botch" them by the time they are sold. We've botched them long before you ever saw an IC with those features. I was working on 14nm SOI in 2012, and the single biggest reason that things don't get out into the wild is yield, but this does not explicitly mean that the transistor did not work. I was making FPGAs because it allowed me to change the routing graph when things fail due to fabrication issues. I was making them asynchronous because at that point, the timing was too difficult to predict on a large-scale. Even by the time I was on the process, we weren't seeing dead transistors, but only mismatch issues. I have had very few dead transistors in my career, and if they were dead, it was often because I made them dead due to playing around with hot-carriers.(As an aside, when you raise the voltage on ICs for overclocking, that's the usual failure) There are many reason that ICs can "fail", but they are not explicit failures. It's usually due to timing because a threshold offset, or in the case of FinFETs, you have two devices per gate, and the mismatch between those can cause timing violations (I have an XOR here). This the origin of "binning" of ICs. Some of faster, some are slower, but they all have the masks to create them. You have test structures that go around different regions of the IC and you get timing information from those.

How do you fix these little nuances? Let's say that you have a cache bank that just doesn't pass timing, you'll just blow a few fuses and your 2MiB cache becomes a 1MiB cache.

If you want the real scoop on this, the IEDM conference is where we talk about how everything is terrible most of the time.

there must be a decent chance of botching some of these.

They're not made separately; the photolithographic process reproduces an entire layer at once, and layers are built up gradually across the wafer. A huge amount of effort goes into ensuring alignment and linearity across the wafer, as well as the use of very high purity materials.

That said, there is still the possibility of failure, and on the smaller processes this can be very high. It's usually a trade secret but here's an article discussing SMIC having a 70% failure rate at 14nm. That results in a lot of discarded failed chips.

Contrary to the other answer, I would not say that "spare" parts are used very often. What can happen is that for some designs which inherently have a lot of duplicated units, such as graphics cards (hundreds or even thousands of execution units), there may be features for disabling the known failed parts allowing the rest of the chip to be used. However, such features themselves take up space. Usually this is "fuses" or "antifuses". It can also be done with lasers.

Chips are subjected to a comprehensive test pattern before they leave the factory - usually this is done as soon as possible, possibly even before sawing the wafer into individual "dice" (using a diamond wire saw).

First, when you manufacture such a circuit there must be a decent chance of botching some of these. How do the plants make sure that they all work?

By putting additional logic on the chips. For example scan chains. These basically connect all flip flops on the chip into one huge shift register, allowing you to program any value (using an Automatic Test Pattern Generator) into your flip flops and checking if the output matches the expected behavior. There are also at-speed tests which make sure the chip/transistors can meet timing requirements. There are built-in memory tests which write and read-back all the memories, checking that the data matches. Tests are done at various stages, some directly on the wafer, some after packaging the chip. In the end you also have integration/system tests after the chip is soldered to a PCB and executes test software.

Second, after prolonged usage surely at least one of these transistors must fail. Does the circuit somehow recognize this and redirect the workflow?

As far as I’m aware this is only done for flash memory (SSDs) where the controller recognizes bad blocks and automatically maps spare memory to the address. Could also be possible for volatile memory.

CPUs and GPUs are sometimes sold with disabled features and/or lower clock speed if certain areas are defective or can’t achieve the highest clock frequency. For example AMD once sold triple core CPUs which were actually quad cores with a defective fourth core.

• Intel's core series apparently does something similar, an i7 that has faulty cores can be downgraded to an i5 or i3. – Crazymoomin Oct 13 '20 at 15:36
• AMD also sold dual core GPUs which were actually quad cores and all cores worked fine :) – towe Oct 14 '20 at 9:10
• There were multiple cases where AMD sold CPUs and GPUs with disabled cores respective shader units and you could re-enable them and with some luck they worked fine. Either they were only defective under special circumstances or AMD fulfilled customer demand by “downgrading” perfectly fine products (thanks economics!). – Michael Oct 14 '20 at 10:04

The simple answer is: To include spare parts on the silicon that can replace the faulty part.

A search for yield and fault tolerance will give you an answer.

Yield = How many faulty vs functional is the factory producing. To improve the yield its a common practice to add spare parts or redundant routes to come around the fact that the silicon has imperfections and the fabrication as well.

Check out these links for more details:

https://www.eetimes.com/self-repair-boosts-memory-soc-yields/

The keyword here is Yield. Manufacturer produces X number of ICs, tests them, and finds out that Y are good. Yield is Y/X.

That's one of the biggest issues with new processes, or when making more complex ICs (adding cores for instance). If the yield is not good, that means your manufacturing cost per functional unit is higher (because you're throwing away a large number of bad ICs) and your production rate (of good ICs) is lower. Altogether not a good thing.

Yields can be as high as > 90%, or as low as 30%. That makes a big difference!

An alternative to throwing away the "bad" ICs is binning:

• This IC has been tested to be 100% functional at the highest speed? It goes in the top-tier bin (most expensive).
• That one only has 3 out of 4 cores which are functional? It goes in a different, cheaper bin, sold has having only 3 cores (the 4th, non-fully-functional, core, is disabled prior to sale).
• This other one only works at a lower frequency? Same thing.

If you decide that your target is to have only 3 out of 4 cores fully functional, then suddenly your yield goes up. This is something Apple for instance have done for some of their processors which were sold as having 3 GPU cores, but actually had 4 on the die, but only 3 were active. That's a sign of a yield issue.

Note that the same applies to many other manufacturing processes, including LCD screens for instance. And of course it's nothing new... Even your fruit and vegetables are subject to the exact same processes!