# Designing Precision Instruments: what to self-calibrate, what to guarantee by design

Context: I am working on a precision instrument (digital output) to measure current to systems, in the range between 300nA and 300mA (120dB). However, I hope that this question can be answered generically enough to be a useful guidance for the design of other precision instruments.

In the design of a precision instrument, I see two principal ways of achieving precision:

1. use precision (low drift, high accuracy) components (amplifiers, resistors, references). Design the instrument so that after initial (offset) trim, offset and gain are precisely what they should be. Precision is guaranteed by design.
2. design in a self-test and calibration section (probably based on precision parts), and during operation of the instrument use the embedded calibration stage to (digitally) account for offset and gain variation. Precision is guaranteed by self-calibration and post processing.

To me, it seems that (2) is not easily possible for a purely analog instrument, while it is likely more cost effective (for the same measure of precision) than (1) for systems that digitize measurements. However, when is (1) the appropriate choice even for digitizing systems?

Which design principle is appropriate in what case, and why?

There is no one right answer and the right answer in any given situation will tend to change with time as different components become available (or unavailable) and costs and various prices change. At one time 0.1% resistors were pretty expensive in smallish quantities, now they're pretty reasonable and available. A 0.01% resistor may cost $15 or so in low quantities- that's only 13 bits of accuracy. If you want to approach the possible 16-20 bits of accuracy with 24+ bit ADC converters it's going to be difficult without some kind of calibration. But you may still need the$15 resistors if you want that calibrated value to stay stable with time and temperature.

The second option is completely practical with analog instrumentation - you just use adjustable, selectable (or selected) or trimmable components. Computers (externally) may be involved. It was once common to use computers to abrasive or laser trim resistor networks in situ to calibrate instruments.

You really have to run through all the plausible options and see what makes sense in a given situation. If there is only a single part that affects precision and it's available in a high volume reasonable cost part, it's often best to just buy that part. For example, an ADC reference. If you only need 0.1% accuracy and you can buy a 0.08% reference that guarantees initial accuracy and has superior stability to the 0.25% part plus trimpot or plus digital gain correction then that may be the correct way to go.

When you have done many instrumentation designs, the number of plausible options will go down as you can reject options that are not likely to be competitive at an earlier stage (one reason why experienced Engineers make a lot more money).

You need to consider the error budget, the desired stability (and other performance characteristics such as noise) the parts cost and overall cost including amortized cost of calibration jigs, procedures, training and in-field recalibration- what happens if an EEPROM gets wiped and your wonderful +/-10ppm calibration suddenly shifts by 5% of full scale error? vs. simply testing to verify that the system with purchased parts is actually working to spec (you definitely should not attempt to omit that step!).

If you are calibrating you can probably hit the center of the tolerance band pretty closely (well, maybe depending on how it's done- resolution of digital calibration still has to be sufficient) at the factory door, so a bit more drift with temperature and time could be allowable and still meet the spec. If you are unlucky and several parts contribute additively to the error budget then you could be near the edge and a small change would put you out of spec. In some cases, stability may be more important than absolute accuracy anyway.

Using digital calibration you can even correct for temperature changes by putting the instrument in a chamber and calibrating it at various temperatures. This is a major PITA and is to be avoided in all but the most extreme situations, in my experience.

• Why the downvote? – Spehro Pefhany Aug 5 '15 at 16:13
• I don't think this answers my question: I still don't understand which method to use when. – corecode Aug 5 '15 at 17:35
• @corecode Someone actually has to go through the design work to figure it out for your individual situation. Design it both ways and see which is best (by whatever criteria you use for 'best'). – Spehro Pefhany Aug 5 '15 at 17:39
• That is exactly a non-answer - what principles do you apply to see what is better? – corecode Aug 5 '15 at 18:23
• For larger runs you will likely find it easier to calibrate your instrument to one expensive reference than hand pick or over constrain the parts. With good design and calibration 0.1% parts with good stability can be trusted with some nonlinearty (which is documented) that can be interpolated to any required precision (up to adc) given enough processing power and calibration accuracy – crasic Aug 6 '15 at 5:54

Spehro's answer is correct. Your question is too broad. There isn't a generic process you can follow to determine which is the best way forward, as the "best" way depends on your criteria such as component availability, cost, time to develop, environment (temp, EMI, vibration, etc.) that the device will be working in, lifetime of the device and so on.

Further, you are asking for a huge measurement range and have not specified anything relating to the project such as budget or resolution. Most people would probably implement this with a number of discrete sensing ranges and switch between them, which adds other calibration concerns.

We discussed this at length on IRC. There is no generic set of engineering principles which will help you get to the "right" answer because the field is so broad and your question so generic. I'm voting to close your question unless you would like to refine it.

It's difficult. Engineering is difficult. If it was easy, everybody would be doing it, and engineers would be paid less. You have to do the work.

Start from an accurate specification of what's needed. The specification might require accuracy over a period of time, or a temperature range, or linearity over a range, or good tracking between the 1x and the 1000x ranges. Does the use case require continuous measurement, or are there pauses in operation? Each of these requirements might push you in one direction or the other.

Note that 'as good as possible' is not a specification. You need to know what your customer wants. Many times I've had to ask the question of a client 'do you really need this spec point here, because if you do, it's going to double the cost?'

General advice. Somewhere, a precision instrument has to go back to a precision reference. Generally, one precision thing is easier to maintain than several. Passive precision (resistor ratios) is easier to get right than active things (voltage references). Time can be made orders of magnitude more precise than voltages and currents (crystal oscillators, PWM, delta sigma).

You can use these resources to calibrate less accurate measurement channels. However, these channels need to be stable (over time, temperature). If you need more expensive components for stability, do you get precision for free at that price?

How long will it take to calibrate these channels? How often do you need to calibrate to compensate for drift? Do you have to interrupt the measurement to calibrate, or can it be done in the background? Do you need two measurement channels and ping-pong them with calibration (like auto-zero opamps do)? Note that calibrating on switch-on is the worst possible time to do it, as the instrument will be used warm, so calibrated at the wrong temperature.

You'll probably find yourself pushed in the direction of having a one high quality voltage reference, one high quality time reference, one high quality resistor to get between volts and current, and a set of ratio resistors to translate accuracy between ranges. However, you'll need low resistance low leakage switches to juggle these resources around, so you may be better off having dedicated precision in multiple places, or use relays in key places instead of CMOS switches.

I don't know about your precise situation. You won't know until you've done quite a lot of detailed work. That work should include prototyping some measurement channels and measuring how they perform. You might will learn something unexpected.