5
\$\begingroup\$

Back in the early 2000s I remember asking about why it was so important that servers use ECC memory. The prevailing wisdom at the time was that systems with lots of RAM would be, statistically, more likely to suffer bitflips. This makes sense - if each cell has a 10-20 probability of suffering a bitflip per second, then 109 cells have a 10-11 probability per second. The more cells you have, the higher the probability of a bitflip in any given time period.

Back then we would be looking at a ballpark of 128MB to 1GB of RAM. These days we regularly put 16GB or more in laptops and desktops, with workstations commonly having 64GB or more. For argument's sake, let's say we've increased total RAM amounts by two orders of magnitude. So we should see a hundred times or so more bitflips, on average, in any given system, assuming that nothing else changed.

The more I thought about it, though, the more I realised that the random bitflip rate should be much higher in newer systems:

  • Lower operating voltages means lower distinction between a 0 and a 1.
  • Lower gate charge means less overall energy required to flip a bit.
  • More densely packed gates increases the likelihood of being affected by cosmic rays.
  • Refresh times don't seem to have gone anywhere. DDR2 tRFC was 40-60 clocks, DDR3 tRFC was more like 90-130 clocks, and DDR4 tRFC is more like 200-450 clocks. When you divide by the internal memory clock rates to get a wall time for each refresh timing it doesn't really show much of a trend - it's effectively flat but with a higher margin either way as time goes on.

But, as far as I know, we're not seeing bitflips everywhere on non-ECC RAM, at least within the confines of our atmosphere.

So, what's the deal? Why aren't we seeing endless bitflips everywhere, at least 100x if not 10000x more frequently than two decades ago? Is ECC actually important in the context of growing RAM sizes, or do the stats not back it up? Or is there some other technology advance that is mitigating bitflip problems in non-ECC memory? I'm particularly interested in answers with authoritative references rather than speculation about error rates.

\$\endgroup\$
  • 2
    \$\begingroup\$ @SunnyskyguyEE75 That's irrelevant to the question and makes an assumption that the probabilistic risk of using non-ECC memory is high in the first place, which is the exact assumption that I'm challenging here. I understand probabilistic risk and impact (my career depends upon it); the question is about whether those risks are being overstated in the first place. \$\endgroup\$ – Polynomial Apr 17 at 13:15
  • 2
    \$\begingroup\$ @SunnyskyguyEE75 I'm not asking how to fix the problem, or how to model probabilistic MTBFs. Just interested in authoritative references as to why a 16GB non-ECC DIMM from 2019 seems to have the same apparent MTBF as a 128MB non-ECC DIMM from 2002, despite the conventional wisdom back then saying "if you have more than 1GB of RAM you should use ECC, because of bit flips" and there being two whole orders of magnitude between the sizes, plus all the other negative factors listed above. If you want to speculate, feel free, but that's not really what I'm looking for in an answer. \$\endgroup\$ – Polynomial Apr 17 at 13:29
  • 3
    \$\begingroup\$ To a great extent it comes down to the use case. The statistical chance of a bit flip (most commonly due to free neutrons) at sea level is really low, but non-zero, however the other part of that is that you may not notice because the bit that gets flipped may be in an area not currently being used. A study by Boeing on servers in Denver established that bit flips (SEUs in the trade) do indeed occur regularly. In avionics we are required to use ECC as the chance increases drastically with altitude. \$\endgroup\$ – Peter Smith Apr 17 at 13:33
  • 2
    \$\begingroup\$ @SunnyskyguyEE75 As I've said, I'm not looking to predict it. I work in infosec; I understand the difference between risk/probabilities and predicting actual discrete incidents. It's practically the same as radioactivity - you can model decays/sec, but you can't predict the individual decays with any certainty. I'm asking about differences between rates. "Ask Crucial" isn't a useful answer here. \$\endgroup\$ – Polynomial Apr 17 at 13:36
  • 3
    \$\begingroup\$ The only company I know of to do a long term study is Xilinx on the Rosetta project. They do publish soft error rates for all their parts. Note that internal geometries and layout (which is used to reduce the neutron cross section) play a large part in effective vulnerability. \$\endgroup\$ – Peter Smith Apr 17 at 13:38
4
\$\begingroup\$

Single-event upsets (SEU) at sea level tend to be caused either by radioactive contaminants in the IC manufacturing materials (particularly the metals) generating alpha particles or by high-energy neutrons (caused by cosmic rays in the atmosphere) ionizing atoms in the silicon itself.

Over the years, manufacturers have greatly reduced the threat caused by radioactive contaminants. There are also proprietary approaches to cell layout that can help to mitigate the risk of SEU. All of this is probably going to be trade secrets and not public information.

And, no, I'm not going to do your literature search for you. However, I recommend that you go through the IEEE Transactions on Nuclear Science if you want references.

\$\endgroup\$
  • \$\begingroup\$ Are saying these are the only significant causes of correctable errors ? Or uncorrectable errors or both or just limited to one cause of the SEU \$\endgroup\$ – Sunnyskyguy EE75 Apr 17 at 15:04
  • \$\begingroup\$ Building on this, for the cosmic ray induced SEU case, the probability of getting hit with a cosmic ray correlates to die area, not number of bits within that die, and in general, memory die area has actually been going down over the years. \$\endgroup\$ – Nate Strickland Apr 17 at 16:56
1
\$\begingroup\$

The answer is, more work needs to be done, and they aren't sure:

The results show that the radiation susceptibility has actually improved somewhat for devices that have advanced to the 0.13μm level, which contradicts earlier predictions. This trend is encouraging, but it may not necessarily continue for devices that are scaled below 0.1 μm. It is important to note that the recent computer modeling calculations for SEE susceptibility of scaled devices predicts a large increase in collected charge that is directly in conflict with test results for heavy ions and neutron soft errors.
Source: The Effect of Device Scaling on Single-Event Effects in Advance CMOS Devices

enter image description here
Source: The Effect of Device Scaling on Single-Event Effects in Advance CMOS Devices

The actual mechanism for SEE's/SEU's has more to do with voltage and geometry than size. The one would think that smaller size and smaller charge per memory element would make memory elements easier to flip and cause errors, but the effect is small and is more related to voltage and geometry. Which is good for space applications that rely increasingly on commercial technology (like cubesats).

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.