# Ambiguity in ADC conversion of analog voltage to digital value

I have question about the common formula used to turn analog input voltage of ADC into an integer binary number that is as below:

I think that in the equation above, N is not an integer but floating-point value. We can not consider [] as an integral part function in this formula.

How is it turned into an integer number? Do we have to use ceiling or floor function?

According to the graph below, I guess that we have to round it to the nearest integer number.

For example if the N is 243.8 we should round it to 244 and if it was 243.2 we should round it to 243. We should not use ceiling or floor function, right?

Edit: To better explain what I mean, I added this better demonstrating plot with its correspondent quantization error graph. Based on the quantization error plot, I guess the rounding function should be used instead of floor or ceiling. I mean that if every voltage value in range of [0, 1/8) was converted to 000, and every value in [1/8, 1/4) was converted to 001 and so on, then using floor function was correct, but obviously some rounding is done, right?

• N is an integer here Jun 15, 2020 at 12:26
• So do you mean that [] is integral part function? But i don't think so, because the fraction is normally something between 0 and 1. so the floor or integral part function of that is zero and ceiling is one. then the N is either 0 or pow(2, n) - 1. right? Jun 15, 2020 at 12:31
• you're overthinking it. [] are brackets, drawn as square for the height. Given the division, they are in fact redundant, so they're only there, ironically, for clarity. In practice, given the other errors in a practical ADC, it doesn't matter whether you round, or take the ceiling or floor, the difference between those is smaller than the inherent errors in the ADC. Jun 15, 2020 at 12:44
• Yes you are right but basically i just want to know which one is correct in theory, and as you said practically there is more error type with greater impact on ADC result, but i have to know which of rounding or ceiling or floor is correct in theory. Jun 15, 2020 at 12:49
• @A.R.S.D. : In the past, I had a long discussion regarding the subject, with TI engineers, including also Bonnie Baker and John Davies. To be short: Please, take a time to read my response here: electronics.stackexchange.com/a/387507/22676 Jun 15, 2020 at 19:43

The "[]" is indeed integer part function. Although, the formula, as you also pointed out, seems incorrect. The output of ADC is an integer, which when multiplied by least-significant-bit (LSB) gives closest approximation of the analog input. Thus, $$N = \left[ \frac{V_{in} - V_{ref-}}{V_{LSB}} \right]$$

Substituting the value of $$\V_{LSB}\$$, $$N = \left[ \frac{(V_{in} - V_{ref-})(2^{n\_bits} - 1)}{V_{ref+} - V_{ref-}} \right]$$

Thus, if you also move your first term inside the integer-part function. You will get the right result.

• But this way your using floor function which i think is incorrect with respect to provided plots, I suggest the right function that should be used is rounding function as we can see that the voltage is converted to the nearest voltage level and this is not always lower voltage level, right? Jun 16, 2020 at 14:24

As comments have pointed out, ceiling or floor is irrelevant in practical applications. The error of the ADC will eclipse any accuracy gained in bit rounding. Offset error, gain error and integral non-linearity make such considerations theoretical at best. Observe the following graphic, courtesy of Microchip:

For theoretical purposes, however, precision is everything. The equation writer assumes you will apply the rules of binary and significant figures correctly. They were probably an engineer. The result of the multiplication, following the rules of significant figures and decimal places, will give you an integer value. Continuing on the sig fig track, if you get a result of 0x56 for example, you might as well add +/- 0.5 on the end. Where does this precision limitation come from? The theoretical ADC itself! Forget rounding entirely. 4.6 and 5.1 are contained within 5 with a +/- 0.5 uncertainty.

Someone might raise the point that a theoretical computer value like 0x33 has infinite precision in a theoretical system, and they would be correct. However, analog-to digital converters suggest lossy conversion, just like going from 3D to 2D loses information. The equation you provided is not a conversion from analog to “digital”, it is a scaled projection onto a range. There are a lot of other things about the equation that have to be assumed as well, such as that negative binary values are not allowed and will clamp at zero or that values greater than 2^N-1 will clamp.

As written, yes I could end up with a decimal value of -34754863894389439842854843868385382809211111.8493629295749383 if I select the write parameters. To represent semiconductor shenanigans correctly in pure theoretical mathematics, a lot more notation would need to be added to the provided equation, but we don’t even like Maxwell’s equations as engineers, preferring abstractions wherever possible. That kind of persnickety math is exactly what the engineers who wrote that equation were trying to avoid, and to be honest, if they didn’t do so, I don’t think ADCs would be around for us to have this discussion. Mathematics is all made up; we might as well make it up useful.

So your result is neither floor nor ceiling. It’s the same as any digital result: a less precise (abstracted to use a computer term) version of a real-world quantity, just like a pixely picture of a person.

I viewed and checked the answers, but unfortunately they didn't thoroughly satisfy my questions and doubts, so I decided to post what I reached to and conclusion.

In theory where the only error that could be faced is quantization error the more precise way is to use rounding.

In practice these considerations are not taken into account not because the effect of quantization error is negligible with respect to other errors (as some mentioned) but because this does not help to reach a more precise model, in my understanding. To better explain it, I posted ADC error types listed in the figure:

As it is clear rounding when there is practical ADC which afflicted by either linearity or missing code error, it does not help to make more a precise model of ADC behaviour as the ADC does not convert input voltages to digital levels which are closest.

These kinds of errors affect the theoretical model which lead to inaccurate calculations, so rounding, flooring or ceiling functions could not lead to best precise model of ADC in practise as it was possible in theory by using rounding.

Ambiguity can be in value of $$V_{shift}$$ in
$$N = Int[\frac{(V_{in} - (V_{ref-} + V_{shift}))*{2^n}}{V_{ref+} - V_{ref-}}]= Int[\frac{(V_{in} - (V_{ref-} + V_{shift}))}{q}]$$ where q is quantisation step
$$q= \frac{V_{ref+} - V_{ref-}}{2^n}$$ $$V_{shift} = [{e.g.-2,-1,0,+1,+2}]*\frac{{q}}{2}={shift}*\frac{q}{2}$$ Values for shift e.g. -2,-1,0,+1,+2 are discutable. $$for\quad V_{shift}=-\frac{q}{2}\qquad V_{in}{≈}V_{ref-}\iff{N=0}$$ $$\qquad for\quad V_{shift}=+\frac{q}{2}\qquad V_{in}{≈}V_{ref+}\iff{N={2^n}-1}$$ As seen from this graph (example for number of bits n=3):