If you have a blackbox that gives you nanovolt signal in the output, how would you collect and process it? Signal to noise ratio is very low. How do you filter the noise? Let's say the frequency range is 0-80Hz in that application. What amplifier would you use? what type of circuit? Thank you
Getting less than 100-200nV in a frequency range from, say, 0.001Hz to 80Hz requires the use of synchronous demodulation techniques. The best monolithic op-amps (eg. LT1028) are around 0.9nV sqrt(Hz) voltage noise alone above their corner frequency so about 8nV for 80Hz BW, but their typical noise corner frequency is several Hz, so the 1/f noise will dominate unless your actual requirement is more like 20-80Hz. The current noise in the white noise range is around 5pA/sqrt(Hz) so it will cause equal contribution at 180 ohms source impedance (and the 1/f corner of current noise is actually well above 80Hz typically so the total noise will be much higher). There is also Johnson-Nyquist noise in the source resistance which can contribute significant amounts of white noise - at room temperature a 1000 ohm resistor has about 137nV RMS noise in your 80Hz BW.
An LT1028 has only 35nVp-p typical (90nV maximum) noise from 0.1Hz to 10Hz, so it's a possibility if you can relax the low frequency bandwidth requirement- it will be about double that from 0.1Hz-80Hz. That's p-p not RMS.
If you can't relax the low frequency requirement, most of what you're concerned with will be called 'drift' since it's below the 1/f corner frequency corners of voltage and current noise of the amplifier. So called zero drift monolithic amplifiers are made with CMOS techniques and tend to have relatively high noise at higher frequencies, typically over 1uV over the 80Hz BW, but little or no 1/f noise. So 100-200nV is not so easy without lots of 1/f noise!
For such requirements, one approach is to try to modulate the source and use synchronous demodulation techniques (amplify, filter over a narrow band, use a phase sensitive detector, and low-pass filter) on the entire signal chain. In a lab the instrument that does this is called a "lock-in amplifier" - a rack mount instrument. The idea is to shift the signal bandwidth up out of the 1/f domain into the white noise region so that the noise over your 80Hz bandwidth will be minimized.
A skilled designer can do significantly better than the monolithic parts (usually at a high cost in power, cost, input impedance, complexity and so on), but even so there are limits. For example, if the source impedance is low enough we can parallel 100 transistors at the input and get an order of magnitude improvement, in theory. There are other less brute-force methods such as running the transistors at higher current.
For the most extreme requirements, cost and convenience no object, I would recommend using a low Tc (4.2K - that's Kelvin, not K ohms) SQUID. It's possible to get measurements with noise in the sub nV range, with extreme care. The resistance of the wires coming out of the cryostat will probably dominate the noise.
Even at nanovolt levels, and with (relatively) low SNRs, there are many established receivers/amplifiers that can give you an acceptably amplified signal without introducing too much noise of their own. Such "small signal amplifiers" and/or "low noise amplifiers" are commonly used in highly sensitive radio recievers.
Unfortunately, however, using frequencies in the 0-80hz reagion will place the inescapable (nearly) 50-60hz "mains power buzz" solidly within your frequency window. In order to reliably detect such small signals in a frequency window shared with THAT particular elephant, you may have to take "extreme measures."
This is possible with moderate quality modern op-amps. The 50/60Hz noise will be a small issue though... I would recommend using a multi stage differential amplifier configuration. The first stage can buffer and amplify a bit, then create a 50/60 Hz notch filter with gain if possible on the second stage.
Some good engineering advice here, but before you worry too much about an actual design, you have to answer two questions: "What do you know about the signal?" and "What do you know about the noise?" So far you've said that the signal is in the range of 0-80Hz. If all the information you care about is contained within that bandwidth, then of course the first thing you want to do is reject anything outside of that. But what else do you know about the signal, and what are the characteristics of the noise within that bandwidth? If there is no fundamental difference between the signal and the noise, then all of the circuitry in the world won't help you tell them apart. Is the noise purely random (i.e., thermal), and does the signal have some structure that would allow you to distinguish it from pure randomness? Is the noise inherent in your black box, or some form of interference (e.g., 50/60Hz hum)? Typically, at nanovolt signal levels, in any real system the answer is "all of the above." That is, there will be multiple sources of noise, each with its own characteristics, and you will have to do battle on multiple fronts.
As you said yourself in one of your replies, "for example if you connect it to the oscilloscope how would you recognize which one is noise which one is the actual signal?" You may not be able to recognize it visually, but if there is a theoretical distinction between the signal and the noise, you need to figure out what that distinction is and that will determine how you attack the problem. If this were a theoretical exercise and you had no idea what function the black box was performing, then there would be no basis on which to say that anything appearing at its output was either signal or noise. If it's a real thing, then think about what it is supposed to be doing, and that should give you some idea of what to expect at its output.