# Questions tagged [precision]

Use this tag to indicate that the topic is related to achieving the highest precision for a circuit. Examples include using 1% resistors instead of 5% or 10%. This tag also can be used in relation to measuring precision in a finished circuit.

123 questions
Filter by
Sorted by
Tagged with
411 views

### Mounting DIL8 package on SO8 footprint

As you might know, it is hard to get components these days. That's why we engineers have to get creative sometimes ;-) The company I'm working for has a shortage of a Zener-type voltage reference ...
• 6,046
94 views

### Noise of a precision/active rectifier

I am trying to build a precision rectifier circuit to implement AM demodulation in a sensor signal chain. For now I have calculated the sum RMS noise of the total signal chain except that of the ...
• 31
69 views

### Opamp output clamp

During power-up, U2A's voltage will be undefined. I would therefore like to clamp it with a relay. Additionally, U2A is buffering a voltage from a 20-bit DAC -- therefore I need to care about thermal ...
• 279
155 views

### Measure capacitors in parallel and series to increase precision?

I measure a capacitor $C_1$ with an instrument that has a tolerance of measurement $\pm 5\%.$ I do the same with another cap $C_2$. Then I put $C_1$ and $C_2$ in parallel (values add up) and ...
1 vote
124 views

### Sigma-Delta differential ADC for PT100 without precision Iexc or Vexc

We are discussing a way to precisely measure PT100 (variable resistance) using a MSP430 with 24bit sigma-delta differential ADCs without having a precision source of current or voltage. We came up ...
• 38
1 vote
77 views

### Options for tracking upsets in very fine movements

I'm working on a small engraving device that sometimes hits a tougher part of the target material. Since the tip is very small and spins at a high speed, the engraver sometimes involuntarily slides ...
• 149
76 views

### Choosing transformer for precision power supply design [closed]

I am trying to figure out what specification transformer I need to design a linear precision power supply which has 3 channels operation with certain requirements. All the 3 channels have total power ...
74 views

### Low spread high power laser

I am looking for a laser which can travel about 10 km through air from one end of a lake to another. It needs to have a minimal spread, i.e. a small diameter spread over the 10 km - less than 10 cm ...
• 121
274 views

### Precision current source 1 nA circuit analysis

I couldn't find any detail explaining this circuit. How would I change it to make it operate between 1 uA to few microamps? Would change 10 Mohm to 10 kohm would greatly degrade the performance? I can ...
1 vote
56 views

### MUX Pulldown resistor Questions

I have a precision application where I am demodulating two sine waves using an analog mux (ADG613) and then Low pass filter to measure it's amplitude. The ADG613 is a 1:4 MUX made into a DPDT (chosen ...
• 479
294 views

### creating a differential amplifier for low voltages (current shunt)

I'm trying to create a "precision" fully differential amplification stage, to measure uA on a 10R shunt, and mA on a 0.01R shunt. shunt resistance values are set at those values for burden ...
1 vote
58 views

### Active voltage limit circuit

In my application there is a signal from 0-10V. The signal gets into 4 different OPAMPs with different gains which are supplied from +- 5V. I would like to limit the input voltage to the OPAMP to 0-2....
• 11
25 views

### MEMS-FPI tunable filter

I've got to control a tunable filter on a MEMS-FPI C14273 IR sensor. It works by inputing a voltage on a range from 25 to 48 V, wich makes the lens move. It needs to be precise because if i go over ...
1 vote
52 views

### How do I deal with Seebeck Effect while measuring resistance with 4-lead ohmmeter?

I'm trying to measure the DC resistance of an inductor I made out of AWG 11 wire. From my calculations its impedance should be around 0.0012 ohm, and the ohmmeter measured precisely 0.0012 ohm. But ...
• 623
6k views

### Is there a way to average resistors together to get a tighter overall resistance tolerance?

I have a very sensitive, kHz frequency level, application where I need two matched resistors of the same resistance better than 0.05%. Like maybe by one magnitude (0.005%). I am basically needing to ...
• 479
107 views

### Op Amp Selection Help for Rail/ Rail IO

I am looking for an operational amplifier which will fit my requirements. My circuit consists of two cascaded non-inverting amplifiers.I have a classical non-inverting setup.The minus pin of both ...
309 views

### Precision amplitude 600kHz square wave generator

I'm looking for a circuit that creates a precision amplitude square wave of 600kHz (50% duty cycle). The amplitude of the square wave needs to be 3Vpk-pk +/-10mV, but can be DC offset. The output of ...
• 1
657 views

### Input voltage drop in LM358 with "an improved precision rectifier circuit."

I found the input voltage drops around 2V drop (observed from oscilloscope) when I connect the ion mobility spectrometer (IMS) signal to the circuit "an improved precision rectifier circuit",...
118 views

### Floating point representation- Double precision

The difference between 201 and next larger double precision number is 2^P. If IEEE double precision format is used then the value of P is _________? -- I know how number is stored using double ...
1 vote
722 views

### How can I make this Component Tester more accurate?

EDIT: In the answer by Reroute (thank you, Reroute!), he references a very active 253 page EEVblog topic -- \$20 LCR ESR Transistor checker project that probably includes exactly what I am seeking. I ...
• 1,999
1 vote
992 views

### Op amp maximum input voltage

Many Precision Rectifier Single Supply Op Amp circuits feed the input AC waveform into the non-inverting (+) terminal of an op amp e.g. see this schematic: Image source: Precision full-wave signal ...
100 views

### Bit growth handling for complex multiplier using Gauss' method

I'm trying to predict/handle the bit growth that occurs when implementing a complex multiplier using Gauss' method. I am looking to implement this multiplier in VHDL. The FPGA I use has a limited ...
347 views

### Arduino frequency measurement to 2 or 3 decimal places

I am in the process of building a Proton Precession Magnetometer to calculate the earths magnetic field strength by measuring the precession frequency of protons in distilled water. I am using an ...
287 views

### What are the effects we need to consider when transmitting extra low voltage (0.1 uV) signal over ~1m long cable?

What are the effects we need to consider when transmitting extra low voltage (0.1 uV) signal over about 1 meter of cable, either coaxial or shielded twisted pair ? The source is low impedance (~5 ...
• 73
67 views

### high accuracy analog differential signal switching solutions to prevent distortion (relays?)

first let me define the application and circuit. the application of this is to sence currents ranging from 1nA to around 900mA. to accomplish this two resistors are used, a 10K for the nA range, and a ...
4k views

### mV variations even with a simple voltage divider

edit: After using a prototype pcb instead of breadboard, placing all possible electromagnetic sources away and unplugging all the equipment from the same line, my noise dropped 2 orders of magnitude. ...
• 79
528 views

### How does oversampling improve the measurement precision?

Regarding improving the precision of a measurement, I have heard that a measurement precision can be improved by filtering or oversampling. When I think of filtering I can make sense because ...
• 3,420
425 views

### Confused with the effect number of decimals at the output of transducer

There is an ultrasonic device which measures and outputs both wind speed and direction and has a very long manual. But to keep it short and summarize, the accuracy and the resolution for the wind ...
• 402
120 views

### Use a precision opamps or instrumentation amplifier for load cell application

I'm making a low power weighing scale with good precision. I'm contemplating if it'll be better to build a instrumentation amplifier with discrete OPAMPs of LPV821 which is low power (Iq 650 nA), high ...
• 310
361 views

### Questions about LM741 in an ECG Circuit Schematic

I'm doing an ECG circuit in which I want to use LM741 for active filters and the Right Leg Driver. Basically I will change the LT1007 in the schematic I used for simulations above with LM741 in real ...
• 1
2k views

### Very high precision zero crossing detection

I have a setup that is equivalent to a guitar string with pickup, with signals of similar frequency range. I have a simple zero crossing detector with a comparator and then just count the interval ...
• 559
110 views

### How to obtain a high precision mV signal within a specific interval using a variable resistor?

I want to design a circuit which outputs a signal ranging from 0-70 mV using a variable resistor ranging from 80.51 Ohm - 183.05 Ohm and a supply voltage of 12 V. My idea so far has been to make a ...
2k views

### Should we use wheatstone bridges nowadays?

For measuring strain gauges, the Wheatstone bridge is the historical choice. In a quarter Wheatstone brigde ($350 \Omega$ typically (?)), with a high input impedance amplifier, and some bridge ...
• 1,274
1 vote
1k views

### Any way to build Analogue Multiplexers using MOSFETs or JFETs or BJTs

Here's my requirement: I need an analogue switch that can do SPST/DPST/DPDT with very low On-Resistance (<10mOhms). There would be current flowing through this switch, albeit in the range of a <...
7k views

### Why isn't there any 9.5 digit multimeter or higher?

Why can't you buy a 91⁄2-digit multimeter? Isn't there any need for it? Is a 81⁄2-digit multimeter the latest high end you can buy? I have tried Keysight, Keithley, and Fluke, but there is nothing ...
• 167
33 views

### Measure precisely a resistance value [duplicate]

Basically, I'm tasked to study the electrical properties of different screws (all in an unspecified titanium alloy, some fully anodized, others only the head) based on previous measurements that ...
2k views

### Fluctuation in resistance value

I'm trying to measure the resistance of a surgical screw that seems to have a low resistance, around maybe one ohm. I use a 1908P multimeter and wires having for resistance around 0.005 ohm. Since ...
1 vote
86 views

### Higher precision with less digits?

Consider an ideal digital milliammeter (zero resistance, zero uncertainty on AD converter input) with realistic (noisy, hysteresic, etc.) AD converter output and a $3½$ -digit display. Often when we ...
• 21
124 views

### Paralleling Multibit current DAC: Improving Accuracy by Averaging

No doubt that if I put in parallel n DACs, multibit (i.e. 16 bit) current output R2R ladder chips, I get n-times higher current at the summing node and the output impedance is also reduced by "n". ...
• 371
96 views

### {enc28j60} Is the 50ppm crystal good enough

I have some 25MHz 50ppm crystals but I am not sure if they are good enough to be used with ENC28J60.
• 2,290
122 views

### Significant DC current on precision DMM despite not being connected to anything

Disclaimer: I'm not a EE professional (I have a background in physics though), so please bear with me if the question seems banal, obvious or just plain stupid. We have a Keysight 34465A DMM, which, ...
• 133
76 views

### Determining equivalent resistance to find the cutoff frequency

I am designing a low noise circuit for a precision application and require a stable voltage supplied to the circuit. I have determined that the MAX6250 voltage reference best suits my circuit. A ...
305 views

Can a low-cost, low-resolution RDAC and a precision voltage reference by used to make a coarsely adjustable high-precision DC voltage reference? Consider for example the AD5121 7-bit RDAC with ±5ppm/°...
• 471
111 views

### Measurement of high voltage with low voltage circuit

Is it possible to make a high-impedance, precision measurement of a 10V DC signal (dynamic range requirement is only ±1V with a DC-1kHz BW) using amplifiers and an Sigma-Delta ADC supplied only by +3....
• 471
472 views

### Low-pass Filter with zero drift amplifier

Can an auto-zero or chopper amplifier be used in a low-pass filter to remove a high frequency carrier? I am trying to design an active low-pass filter with a 1kHz cutoff and very low DC drift, offset, ...
• 471
697 views

### Is this normal for a 12 bit Microcontroller ADC (STM32F303)?

I am using an STM32F303 with a 12bit ADC to read an output voltage using the ِADC itself.. as far as I know the Vref of the ADC is the same as the VDD of the device. Am I right? However, when feeding ...
1 vote
108 views

### Dictionary: is PWM precision the same of PWM resolution?

I have an issue in identifying the meaning of PWM precision. I've always talked about PWM resolution, and to me seems to be 1/2^(n-1). Is the precision of the PWM just prequency precision? Or is the ...
• 1,228
307 views

### Is the HP206C really so precise?

I'm looking for barometer sensors for use in Arduino projects. One of the items I consider is the HP206C (at Reichelt, Germany). The vendor contradicts himself a little bit when he mentions 0.1 m and ...
1 vote
292 views

### Would a current feedback op-amp work well as a precision ground reference?

I've been looking into precision ground reference circuits to form ±5V rails from a 10V supply for a project I'm working on, and noticed that current feedback (CFB) op-amps have some desirable ...
• 1,291
1 vote