How are new 'best performance in the world' devices measured?

Question

Say you build an ADC or a DAC (or a scope/spectrum analyser) that has the best performance in the world. With what exactly do you test it with?

I'm guessing:

1. Build the best ADC. You'll get digital numbers that you can collect, then run them through an FFT on your computer. But how do you know if your source or your ADC is the weak link if you get disappointing numbers?
2. Build the best DAC. As long as you've a better ADC, you can use the ADC to measure the DAC.

Am I thinking along the right lines?

Answer 1

If you're designing a circuit that measures some quantity, you'll ideally want to relate that measurement back to some absolute physical quantity. This is how NIST and other national standards bodies create the "standards" used to calibrate laboratory test equipment.

The details of how you might do this depend dramatically on exactly what it is your circuit is measuring.

To give an example, one of the conceptually simplest cases would be if you developed a timer circuit that is meant to accurately produce an output pulse once per second. You could relate the accuracy of your timer circuit to the actual definition of the second: "the duration of 9,192,631,770 periods of the radiation corresponding to the transition between the two hyperfine levels of the ground state of the caesium 133 atom". That is, you could use an atomic clock built around a cesium cell to produce a 9.192631770 GHz reference signal, then divide that clock down to produce an absolute 1 Hz output signal that you could compare to your new circuit's output.

Most likely, you would keep your atomic clock in very highly controlled environmental conditions, while adjusting the temperature, humidity, etc., around your new circuit over the ranges you want it to operate under, to show that its performance is insensitive to those environmental influences.

Your example of a new ADC or DAC circuit is more challenging, because a "standard volt" is not easily produced, but must be related to fundamental physical measurements of time and current using the Josephson effect.

Finally, in the worst case, that you are in fact producing a device of the accuracy of the national standards themselves, what you would have to do is convince someone else to independently produce a similar circuit, and then compare the two new devices to be able to put some limits on the errors in either one. My understanding is that several national standards bodies are in fact in the process of doing just that to develop the watt balance as a reference standard for the measurement of mass.

Answer 2

In many cases, there is a huge distinction between "build the most accurate and precise possible piece of apparatus for measuring X" and "build the most accurate and precise piece of apparatus, within certain constraints of cost, size, power consumption, data acquisition speed, resistance to environmental effects, etc." If one is trying to design a practical device which has to meet any sort of tight constraints, and if one can afford to ease some of those constraints on a test-bench setup (since, among other things, one will probably either want to build more than one of the device one will be testing, or one will want to use it in places or circumstances where one wouldn't have to use the bench-test setup) the test-bench setup may in many cases easily be more accurate than the device being evaluated.

Answer 3

Semiconductor companies sometimes publish application notes that explain some of the techniques used to verify the performance of new devices.

For example, from Linear Technology:

1ppm Settling Time Measurement for a Monolithic 18-Bit DAC

775 Nanovolt Noise Measurement for A Low Noise Voltage Reference

And from National Semiconductors:

What's All This Teflon Stuff, Anyhow?