Why can't you just average ADC samples to get more resolution from an ADC?

Question

I'm trying to get more than 10 bits of precision from my Arduino ADC but I can't really figure out the theory behind it. An often quoted Atmel Application note (http://www.atmel.com/Images/doc8003.pdf) says that

It is important to remember that normal averaging does not increase the resolution of the conversion. Decimation, or Interpolation, is the averaging method, which combined with oversampling, which increases the resolution

Then what they propose for 'Decimation' is moving the decimal point. Which amounts to halving the binary reading for every place you move it so you might as well divide the base 10 value by 2 or 4 or 8 or what have you. Am I understanding decimation wrong?

Answer 1

I took a look at the note and that is indeed a weird claim (or a confusing way of saying what they actually mean).

Perhaps what they actually mean is the point that if you want to get more resolution, you can't divide/shift the number afterward to the same scale as a single sample because (in integer arithmetic) that would throw out the bits you gained.

If your ADC samples are noisy, then of course you can divide to get a less noisy value at the original scale.

---

The other thing I thought of from just your question was the point that to do oversampling right you need to use an effective low-pass filter, and a straightforward moving average is not as good at being a low-pass filter as a properly designed FIR (or IIR) filter — but that doesn't seem to be supported by the text of the note.

Answer 2

If you ask someone to measure a 45.2cm board accurate to the nearest centimeter, they would (or should) answer 45. If you ask then to measure it again, they would answer 45 again. Repeat the exercise 8 more times and the average of all the measurements should be exactly 45. No matter how many times one samples the input, one will end up with a value of 45. The average of all those readings would, of course, be 45 (even though the board is 45.2cm long).

If you had the person adjust the measuring apparatus so as to read 0.45cm long before the first measurement, 0.35cm long before the second, 0.05 cm long before the fifth, 0.05cm short before the sixth, etc. up to 0.45cm short before the tenth, then two of the measurements would read 46 and the other eight would read 45. The average of all of them would be 45.2.

In practice, managing to bias things so precisely is difficult. If one randomly adjusts the measurement apparatus before each measurement to read somewhere between 0.5cm long and 0.5cm short, then about 1/5 of the measurements would read 46 and the rest 45, but because the adjustments are random the actual fraction might be higher or lower. Taking ten measurements would not add quite a full significant figure worth of precision, but averaging about 100 would.

I'm not sure I quite understand the paper's rationale for the distinction between averaging and right shifting. One needs to be mindful that the apparent precision achieved by averaging may exceed the meaningful level of precision, but from my experience the question of when and how much to right-shift should be driven by the limits of the processor's numerical range. Working with numbers that are scaled up as much as they can be without causing overflow will generally minimize the effects of rounding errors, provided that one doesn't attach undue significance to small amounts of noise.

Incidentally, in the original usage, to "decimate" an army was to kill 1/10 of the soldiers therein. To decimate the data from an ADC is to discard part of it. The common prefix with the phrase "decimal point" does not imply an association.

Answer 3

The short answer is noise, and its not necessarily the noise that matters, but the type of noise. The other problem is nonlinear effects like INL that throw off the average value

First on to Noise:

If we were to sample a Gaussian distribution it would look something like this:

The red line is closer to the actual thermal distribution (averaged over time) and the blue histogram represents many ADC samples. If we were to continuously sample this distribution we would get better statistics and we would be able to find the average value or mean with better accuracy(which is usually what were after, Yes I realize signals move around, there is filtering and signal to noise depending on the frequency content but lets just consider the DC case where the signal is not moving for now).

$$ \mu = \frac{1}{n} \sum_{i=1}^{n}{x_i}$$

The problem is flicker noise or 1/f noise, it shifts the Gaussian mean around and causes the statistics to break down, because the distribution is no longer gaussian.

This is a poor model but you could consider it looking something like this INL is also a problem because it can introduce a few bits of error which also throws off the mean.

$$ \mu = \frac{1}{n} \sum_{i=1}^{n}{x_i}+error$$

That is probably confusing, lets look at the time domain as shown below

In the top image you can see a signal with gaussian noise it would be easy to "draw a line" through the middle and find the mean. The more sample you have from a signal like this, the better accuracy and knowledge you will have of the mean.

In the lower image you can see what flicker noise looks like, averaging is not going to help here.

The problem is most electronics have flicker noise, resistors do not (assuming there is no influence from the room temp) but transistors and IC's do. There are amplifiers called chopping amplifiers that do overcome these effects.

Another thing to know is there are ADC's (linear has a new SAR core) where the engineers have worked to eliminate the effects of 1/f noise (and other nonlinear effects of ADC's like INL ) to a level much lower than the the ADCs bit value. You can employ heavy oversampling and get out 32-bit values out of a 14-bit core.

Source: EDN- 1/f Noise—the flickering candle

Answer 4

Then what they propose for 'Decimation' is moving the decimal point.

not exactly. the decimation part of it is arguing that, correctly in my view, that the normal "averaging" of multiple samples, but retaining the bit width, doesn't retain as much information. So if you average m n-bit ADC readings, the resulting average is still an n-bit adc reading.

the approach proposed is, to put it mildly, is to average n-bit ADC readings so that the resulting average has a higher bit-width. For example, summing 4 10-bit adc readings and dividing the sum by 2 yields a 11-bit adc reading.

I thought that has always been the way oversampling has been done professionally. This simple averaging by folks on the net is widely understood to be the wrong approach.

the other point that in order to reduce the noise, oversampling is only effective if there is noise is the right one. If you had a 10-bit ADC designed by God (ie every reading is the absolute true reading, with no variation), oversampling wouldn't have worked.

the particular circuitry towards the end of the article about using a pwm to add noise is incorrect: the pin adding noise should have a DC blocking capacitor. and a less substantive point is that it doesn't have to be a PWM pin. A normal GPIO pin would work.

Answer 5

First, an ADC is only as good as its voltage reference. If your arduino uses the +5V as a reference, you can forget about any kind of precision, since the +5V regulator is rather noisy, low accuracy like 1-5%, and its output voltage will depend on the amount of current drawn from it, both at the time of measurement and also in the last few milliseconds.

So, if you need accuracy or precision, please select a voltage reference which matches your requirements. If you don't need absolute accuracy on voltage, it'll be cheaper, since you'll only need it to be stable, instead of accurate and stable.

I have not tested the SAR ADC inside the arduino. I have experience with the one in AT90PWM3B which is a close relative. It is pretty good. With a constant input voltage, you get the same ADC reading, over and over again, with 1 LSB fluctuating if the voltage is between values. Can't expect a better result from a SAR ADC. (I did use a good quality external voltage reference)

So, noise is not a problem here...

In fact, noise is your friend...

Let's suppose the voltage you want to measure falls on ADC value 100.1

You make 10 measurements, but since the ADC is good, you get 100 every time!

So you need a bit of noise on your signal, like one LSB of noise, to ensure if you measure 100.1 then you'll get 100 nine times out of ten, and 101 one time out of ten. So it will average to 100.1, get the idea?

If it comes from a sensor, usually you'll have enough noise for free.

Answer 6

What you are missing is the meaning of "decimation."

Strictly speaking, "decimation" is reduction to 1/10. That is, from 10 reduce to 1.

"Decimation" as used in sampling loses the strict meaning. Rather than 1/10, it means "to reduce in number."

What this means is, that you average some number of samples, and reduce the number of samples by the same number.

As an example, if you sample at 1000Hz and average 4 samples, you only keep the average. At the end, you only have 250 samples per second instead of 1000. You've lost time resolution, but gained one bit of voltage resolution.

For each factor of 4, you gain 1 bit. Average and decimate by 4, and go from 10 bits resolution to 11 bits of resolution.

Another factor of 4 (4*4=16 total) gets you from 10 bits to 12 bits. Another factor of 4 gets you to 13 bits of resolution.

But, note that you are now oversampling by a factor of 64. Your effective sampling rate goes down by the same factor. Using the example of 1000Hz sampling rate, you are down to about 15 effective samples per second.

This is decimation, and it is how many high bit ADCs get their high resolution. They sample at a high rate, average (or use a digital low pass filter) and decimate.

At the extreme end, you have a single bit ADC (a simple comparator) that oversamples by several millon times to give an effective bit depth of 16 bits.

One thing you need to keep in mind for this to work is that you need noise in your signal approximately equal to the smallest value your ADC can measure. For a 10 bit ADC using a 5V reference voltage, that would be noise of about 5mV peak to peak.

Averaging the noise is where the bit gains actually come from. Imagine that you have a signal (DC) that lies exactly between the value of 512 ADC counts and 513 ADC counts. With no noise, the value measured will always be the same - averaging will give you the same value as the samples.

Add noise at just about the smallest measurable value, and it looks very different. Though the signal itself doesn't change, the measured values will "wiggle" around the true value of the signal. The average now is different than the samples, and the more samples you use the closer you get to the real value of the signal

---

I have used this technique with an Arduino (which uses an Atmel processor with a 10 bit ADC) to get better resolution for some measurements I was making.

I got it up to 13 bits, but found I needed more. I could have gone for another factor of 4, but that would have taken too long for each sample and only gotten me one more bit.

The experiments with oversampling showed that what I was doing could work (I got recognizable but noisy results) without having to spend time and money to get a better ADC. With proof of concept, I could go ahead and get that better ADC - and getting that proof only cost me a few lines of code and a little time.

I found I needed at least 16 bits. That would have meant averaging 4096 samples.
That's about half a second using the fastest sampling possible with the Arduino software.

Since I needed 14400 measurements, the full run would have taken 2 hours.

I'm not that patient, and the things I was measuring wouldn't stay constant for that long. I had to switch to using an ADC that uses a much higher oversampling rate internally, and that delivers higher resolution samples at a lower rate.

As with so many things, decimation is compromise that can get you better performance in one direction (bit depth) while costing you performance in another direction (sampling rate.)

Answer 7

So you can study the theory, but i can tell you that in practice only a simple model matters. You can average, as long as you signal is within noise. Then the average of noise component will be zeroed out, while the signal will remain. This way you will get resolution in expense of bandwidth.

If you have for example a 16-bit adc and it's last four bits are noisy, you can filter them and get the signal there. But if you have just one bit noisy, there is not much to average, so you will not get much new information.

If you need really high resolution (and low bandwidth) look how sigma-delta ADC work. They have 1-bit high rate signal that is then filtered down to some bandwidth with higher resolution, sometimes 20 bits and more.

Answer 8

Why can't you? You can, but you must consider all the sources of error and noise to make sure your plan works.

Averaging works to improve the resolution by reducing the standard deviation error, \$\sigma\$. The criteria to meet is that the Gaussian Noise must just exceed the quantization error. The spec to be defined is the total error and make quantization error or resolution contribute only a minor amount of the total error budget.

e.g. If you wanted to improve resolution by 2 bits, but your noise was already 3 bits you must consider how to reduce the noise by 2+3=5 bits while increasing resolution at the same time by 2 bits.

• This could be a digital solution with averaging >25 samples with the cost of latency or an analog solution by noise rejection filtering, balancing signals to reject common mode noise and/or shielding the signal better along with digital averaging.

Where n is the desired extra bit(s) of resolution, then right shifting a binary number (or decimating) x1 is equal to /2.

For averaging it means the \$\sigma\$ of noise is reduced by \$\sqrt x\$ for x samples but also that the latency is increased by x time samples, hence oversampling is needed to reduce latency.

Note the term "decimation" applies to both decimal values and binary coded decimal numbers. You can visualize if you have a counter that reads integer values and then by averaging 10 results, you divide by 10 to achieve an improvement with an extra decimal place but the \$\sigma\$ of noise is only reduced by \$1/\sqrt 10= 1/3.3\$ for x samples but also that the latency is increase by x time samples, hence oversampling is needed to reduce latency,

\$f_{oversampling}=4^nf_{Nyquist}\$

However the noise at that sample rate,\$f_s\$ must be just enough to dither +/-1 bit over some of the x samples, to get the best improvement in resolution.

• For example, if the BW is 10kHz then \$f_{Nyquist} =~\$20kHz
• and if you wish to convert 10 bit resolution to 12-bit then n=2 Increasing the resolution from 10-bits to 12-bits requires the summation of 16 10-bit values. A sum of 16 10-bit values generates a 14-bit result where the last two bits are not expected to hold valuable information.

---

Too quantization error or too much random noise will require more averaging to reduce the error and more averaging increases latency of the result.

To optimize an ADC for speed and error, one must define the total error budget and available resolution (bits), desired SNR or absolute error for any given signal within the entire measurement range. Defining all the sources of error at first may seem difficult, yet necessary including;

e.g. gain error, offset error, quantization error, CM noise error, DM noise error, Vref noise or offset error, environmental noise, etc., latency error (from averaging)

Then determine how many more bits of resolution you need to achieve the above design error budget after all other sources of error have been minimized.

The same applies to averaging ( for slow signals) and oversampling the signal bandwidth and decimating for real-time ADC.

This will not correct for gain or offset errors and if there insufficient random noise then noise must added to dither the signal. Ideally all other sources of noise and error do not exceed 1 bit such that the standard deviation or dither is only +/- 1 value over the number of samples. However , there must be sufficient noise so that the same reading is not obtained by successive samples in both methods.

Answer 9

The theory behind it can be taken, in short form, from this sentence on the Wikipedia article about oversampling:

However, the SNR increases by sqrt(N) (...). Summing up uncorrelated noise increases its amplitude by sqrt(N), while summing up a coherent signal increases its average by N. As a result, the SNR (or signal/noise) increases by sqrt(N). In the example, that means while with N=256 there is an increase in Dynamic range by 8 bits, and the content of "coherent signal" increases by N, but the noise changes by a factor of sqrt(N)=sqrt(256)=16 in the example (not to be confused with an increase of 16 bits), so the SNR changes by a factor of 16.

So, provided your signal is matches certain criteria (like being slower enough and having low noise) you actually do increase one bit for every 4 samples. Then, for every 4 resulting samples, you can get "join" then again to form another, higher-resolution sample, such that in the end you get log_4(n) bits for every n samples you read on your ADC.

As for the decimation part, it is not really averaging, specially if you take into consideration that we're talking about integers here (ADC samples). For example, if you have samples 1, 1, 3 and 2, your average would be:

int result = (1+1+3+2)/4;

Since you're averaging with integer math, your "mathematical result" of 1.75 will round down to 1. If you then multiply by 2, you'll get 2.

Now, if you decimate it with:

int result = (1+1+3+2)>>1;

Your result will be 3. You can argue that this is the same as dividing by 2, but surely you can't argue that 3 is the average of 1, 1, 3 and 2. See the difference?

Now you may be tempted to just sum everything and not discard the last bit. But remember that this bit is noise: you can't really use it.

Answer 10

It sounds like everyone's covered the theory part of your question already, but since you are using an Arduino you might want to read through my adventures trying to boost the ADC's resolution with this technique:

Improving Arduino ADC resolution with Dithering & Oversampling

The character of the noise is a critical part of the story, and it turns out you can generate a reasonably good dither by simply pulsing a pin with a resistor on it while you read the ADC asynchronously. It's not perfect, and you do get a small synchrony offset that varies depending on how many extra samples/bits you are trying to achieve. I'd also accept the criticism that this technique depends on the poor rail stabilization in the Arduino, so is really designing to a flaw, rather than following good practice. But it is very easy to do.