# Calculating the sensitivity of an ADC with respect to resolution

I am reading a text that explains about measurement sensitivity and accuracy. In a part of it it has the following examples (with no explanation!)

3 and 1/2 digits (2000) on 2V range = 1mV

Which makes sence if you do calculate 2/2000 = 0.001 = 1mV

Now it has the following:

4 and 1/2 digits (20000) in 2Ohm range = 100mOhm

I can't get this one, I am calculating 2/20000 that results in 0.0001 which is equal to 100uOhm, Or I am stupid and doing it wrong!?

Link to the original text, You can see the example above on page 3.

Apparently, they have a new version here. I can see they changed "m" to "µ" in newer version, so the problem was not at my side :P

• Do you have a link to the text? Russell is correct about the second sum being incorrect, and is probably just a typo (or font problem) but depending on the context it just might be something to do with the absolute accuracy. Sep 20 '11 at 14:22
• I will upload a copy tommrow. don't have access to it at the moment. Sep 20 '11 at 17:37
• @Oli I have added the link to the text above. Sep 21 '11 at 7:00
• Thanks, the context of the original and the new version confirms it is simply a mistake. Sep 21 '11 at 22:25

We're using HTML here, so we can write "µΩ" directly.

It's probably a font problem as other have said, but there is one more thing to consider. They way resistance is usually measured in a multimeter, the signal to the A/D (or in old ones to the analog meter) is not proportional to resistance. Resolution therefore is dependent on resistance. You can't just divide the maximum value by the number of displayable answers to get a fixed resolution without knowing something about how the meter is constructed.

• I think the example was meant to be a simple division of max value by count to show resolution per count. I don't think things like ADC technology were being considered. As ever, I may be wrong :-). Sep 20 '11 at 15:32
• @Russell: Yes, I think that's what the OP was doing. The point is, however, that this can be very misleading. Displayable resolution is not measurement resolution is not accuracy (as I know you know, this is for the others). Sep 20 '11 at 16:07
• I would expect that most multimeters nowadays would measure resistance by injecting a constant current and measuring voltage. In ancient times, including an extra transistor for a constant-current source would have added significantly to the cost of a meter (which in many cases wouldn't have any transistors at all!) but constant-current-based resistance measurement is so much better than constant-voltage measurement in nearly every way that there's really no reason not to use it. Sep 20 '11 at 16:23
• @Supercat: Not neccessarily. A resistor divider has the advantage of 0 to infinite resistance not exceeding full scale. It is also based mostly on the accuracy of a single resistor. It's not as cheap and simple to make a current source with the same accuracy. As always, the "right" solution depends on a lot of factors and there isn't a single right one. Sep 20 '11 at 16:32
• @Supercat: The change in resolution over the range together with the fact that you don't know what method is being used is precisely my point. In reality, the result wouldn't go into a 12 bit A/D, but a much higher resolution sigma-delta A/D. These are slower, but that's not a problem in a handheld meter. The result is non-linear with the resistance, but that can be easily computed in the micro already in there running the display, debouncing the buttons, and implementing the user interface, etc. You can't assume this isn't happening unless they explicitly say otherwise. Sep 20 '11 at 19:00

2/20000 = 1/10000 = 100 micro-ohm, as you say.

The "problem" is almost certainly with the type setting.

They have used "m" for micro. This may have started off as Greek "mu" which can be transliterated as u as in uF.

BUT I have seen a text where "mF" was displayed BUT when you pasted the text and changed the font it changed to a Greek "mu".

• since they calculated the first as 1 mili Volt I asume the second one I pastes is also 100 mili Volt. So should I take this as a typo error? Can you explain this formula a bit further please? Sep 20 '11 at 13:59
• As I said - it's NOT a formula issue, it's typesetting / font rendering / typo. They show milli as m and micro as m in the font they use. If you pasted the text into another font it MAY come out as uOhm rather than mOhm. In this case mOhm means micro-Ohm. In the prior case mV meant milli-Volt BUT if they had tried to write uVolt it would probably also have shown as mVolt. // OR it may just be a typo by the writer. Sep 20 '11 at 14:45
• Most likely there's an automatic text replacement routine somewhere in the path between the document editor's keyboard and the printer which automatically converts "mF" to "µF", and whoever write the documentation was expecting it to do likewise for mOhm, notwithstanding the fact that "µF" is a far more common unit than "mF" but "mΩ" and "mV" are both more common units than "µΩ" and "µV". Sep 20 '11 at 16:27