6
\$\begingroup\$

This is more of a general question to see which methods are the most common to improve the detection of a signal which is buried in noise. Currently, we are building an optical system for medical imaging and the signal that is being detected is 1000x lower than the noise floor.

Currently, we are looking at methods such as lock-in amplification/detection and other types of filtering but it was an open question to see which are common methods to improve the detection of a signal in a noisy background.

\$\endgroup\$
  • 4
    \$\begingroup\$ More context required. \$\endgroup\$ – DKNguyen Mar 13 at 15:29
  • \$\begingroup\$ Is there any specific information that you would like to know? \$\endgroup\$ – James Mar 13 at 15:32
  • 2
    \$\begingroup\$ My understanding is that where an LIA is able to be implemented then it is liable to be usefully superior to any other method. ie you can only use it where you are generating the signal which is then processed by the system and then can be correlated against the source signal. That's not always possible (eg detecting signals from an unavailable source) BUT when it is you know frequency, source phase, source amplitude (both able to be affected by the target system, stability , .... so detection is vastly easier than in any other case. \$\endgroup\$ – Russell McMahon Mar 14 at 1:55
  • \$\begingroup\$ Should be looking at Hobbs' book: Building Electro‐Optical Systems: Making it all Work onlinelibrary.wiley.com/doi/book/10.1002/9780470466339 \$\endgroup\$ – D Duck Mar 14 at 23:17
21
\$\begingroup\$

The only way that a signal that is "buried in the noise" can be detected is if you can run the signal + noise through some filter that attenuates the noise more than it attenuates the signal. At which point the signal is no longer buried in the noise, so "buried in the noise" was just a hasty assumption.

In radio carrying an audio (or Morse code) signal in SSB or AM, you take the signal + noise and you filter it by the approximate bandwidth of the signal, then you run it through a detector.

In radio carrying digital data, you run it through a matched filter and then a detector.

In spread-spectrum radio, you correlate the signal + noise with a pseudo-random sequence, then bandpass filter, then detect.

In visual systems, you correlate the noisy image with a 2-D prototype of the anticipated signal, or you run the noisy image through a spatial low-pass filter, then you detect.

In all cases, the signal has to be distinct in some way from the noise -- if it is not, then you cannot filter out the noise without filtering out the signal, too.

I'll add to this:

At the top level, a filter for signals is like a coffee filter or a colander: you have the stuff you want (coffee or fresh-cooked pasta) and the stuff you don't want (coffee grounds, or starchy hot water), but it's all mixed together. So you run the mess through a filter. In the case of coffee, you keep the stuff that gets through the filter. In the case of the colander, you keep the stuff that gets left behind. In either case, you're using the fact that one thing (coffee grounds or pieces of pasta) is bigger than the other (water molecules and all the other stuff you want in coffee, and don't want in pasta).

A signal filter does the same thing -- you get rid of what you don't want because it is different from what you do want. If you can't figure out how it's different, and how to build an algorithm to separate it -- you can't filter your signal from your noise.

| improve this answer | |
\$\endgroup\$
  • 2
    \$\begingroup\$ A lock-in amplifier "sort of disobeys" the need to filter signal from noise. Sort of. By correlating signal with known matching reference signal you extract correlated components without first removing all noise. Yes? \$\endgroup\$ – Russell McMahon Mar 13 at 23:43
  • 2
    \$\begingroup\$ A lock-in amplifier correlates the signal and then low-pass filters the result. The correlation step turns the signal into DC, the filter gets rid of noise without getting rid of signal. So it's pretty much doing exactly what I'm talking about. \$\endgroup\$ – TimWescott Mar 14 at 0:23
  • 3
    \$\begingroup\$ OK. As per my "sort of disobeys". I'm not wholly comfortable with the assigning of standard terminology to actions which are extremely usefully different to their normal usage in the same context. The correlation step "turning the signal into DC" (effectively synchronous amplitude detection) is not available to most other methods, and the noise is much more irrelevant than in most non phase lock methods. Closest comparison is a PLL which has a finite bandwidth within which it will seek for signals which self correlate over time. ... \$\endgroup\$ – Russell McMahon Mar 14 at 1:51
  • 1
    \$\begingroup\$ ... The LIA has essentially zero bandwidth without conventional filtering (but adding conventional filtering helps) and as you know EXACTLY the frequency of the target signa l the filter canm be as narrow as overall requirements dictate. I know you know all that (except any parts where I'm wrong :-) ) - I'm not arguing process - just how to best see it. || Where an LIA is able to be implemented then it is liable to be usefully superior to any other method. \$\endgroup\$ – Russell McMahon Mar 14 at 1:53
9
\$\begingroup\$

The general concept of detecting a signal in noise is knowledge of something about the signal, and ideally something about the noise.

The easiest thing to use is spectral knowledge. If you know the signal occupies some part of the spectrum, then you can safely filter out noise in other parts of the spectrum without losing signal. This is taken to its extreme in the 'lock-in amplifier', which is basically just a method of creating a very narrow bandpass filter at precisely the frequency of the signal.

A more general property of the signal is its waveform. We can correlate the signal plus noise with a copy of this waveform, and then average. The noise does not line up with the correlating waveform so adds as power. The signal does correlate, so adds as voltage, leading to a 3dB improvement in SNR each time the number of averages is doubled.

| improve this answer | |
\$\endgroup\$
7
\$\begingroup\$

Another technique that might be useful is autocorrelation

Autocorrelation, also known as serial correlation, is the correlation of a signal with a delayed copy of itself as a function of delay. Informally, it is the similarity between observations as a function of the time lag between them. The analysis of autocorrelation is a mathematical tool for finding repeating patterns, such as the presence of a periodic signal obscured by noise, or identifying the missing fundamental frequency in a signal implied by its harmonic frequencies. It is often used in signal processing for analyzing functions or series of values, such as time domain signals.

Different fields of study define autocorrelation differently, and not all of these definitions are equivalent. In some fields, the term is used interchangeably with autocovariance.

| improve this answer | |
\$\endgroup\$
5
\$\begingroup\$

And then there is the idea of averaging multiple independent observations. In overly simplified terms, the aim here is to increase the signal strength while letting the noise cancel itself out, i.e., the desired signal increases faster than the noise, and the more samples you average the better.

| improve this answer | |
\$\endgroup\$
1
\$\begingroup\$

If you know exactly what the signal looks like, in the time domain, you can implement Matched Filters to discard noise energy in frequency regions that are not needed to construct the waveform.

| improve this answer | |
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.