# What does "correlation" mean in signal processing?

What do the words "correlated" and "uncorrelated" mean in signal processing? E.g. - "uncorrelated white noise.."

What it usually means:

"correlation, In statistics, the degree of association between two random variables. The correlation between the graphs of two data sets is the degree to which they resemble each other. However, correlation is not the same as causation, and even a very close correlation may be no more than a coincidence. Mathematically, a correlation is expressed by a correlation coefficient that ranges from −1 (never occur together), through 0 (absolutely independent), to 1 (always occur together)."

(from Encyclopedia Brittanica)

Uncorrelated white noise means that no two points in the noise's time domain are associated with each other. You can't predict any noise value at any other time from the noise level at time $t$. The correlation coefficient is 0.
Even if you know the noise signal over an eternal time, except for that one picosecond, all this information can't help you to fill in that picosecond's level. That's zero correlation.

Correlation within the signal itself is called autocorrelation.

• The last sentence in the quotation from Encyclopedia Britannica is incorrect in that if the correlation coefficient $r$ has value $\pm 1$, the two quantities $X$ and $Y$ are said to be perfectly (positively or negatively) correlated. In fact, $Y = aX+b$ exactly with $a>0$ and $Y$ increasing with $X$ if $r=+1$, and with $a<0$ and $Y$ decreasing as $X$ increases if $r=-1$. For $0<|r|<1$, $Y\approx aX+b$ with the approximation improving as $r$ gets closer to $1$, and same relation $\text{sgn}(a)=\text{sgn}(r)$. Jun 12 '12 at 12:45
• @DilipSarwate, From the phrase "never occur together", etc., we can imagine that the Brittanica author was writing about random variables that only take two values indicating occurence or non-occurence of some event. Nov 30 '12 at 17:32
• @ThePhoton Even restricted to random variables taking on values $0$ and $1$ only indicating non-occurrence and occurrence respectively, my interpretation of the phrase "never occur together" is that $P(1,1)=0$ while $P(1,0),P(0,1)$ and $P(0,0)$ can be nonzero. However, $r=-1$ only if $P(0,0)$ also equals $0$. That is, when $r=-1$, $P(0,1)$ and $P(1,0)$ both are nonzero (they need not be equal) and $P(1,1)=P(0,0)=0$. Equivalently, $r=-1$ if and only it always so happens that exactly one of the two random variables has value $1$ and the other has value $0$ Nov 30 '12 at 18:33
• @DilipSarwate, OK, now I get it, and I agree the Britannica language is not as precise as it could be. Nov 30 '12 at 18:48
• "Correlation within the signal itself is called autocorrelation" then What is cross correlation?
– engr
Apr 26 '20 at 7:36

Uncorrelated white noise is a pleonasm in the sense that there is no such thing as correlated white noise. One either has white noise which by definition has certain properties including a lack of correlation, or one has noise that is correlated and so cannot be described as white noise in any sense of the phrase.

The mathematical model of continuous-time white noise is a convenient fiction that accounts for the physically observed fact that the noise power spectrum at the output of a filter with transfer function $H(f)$ is proportional to $|H(f)|^2$. If we pretend that the input to the filter is white noise -- which has infinite bandwidth, and flat power spectrum over this infinite bandwidth (and hence infinite power) -- and apply standard random process theory, we come to the result that the noise at the filter output is indeed proportional to $|H(f)|^2$. So this infinitely powerful mythical beast white noise is a plausible explanation for our physically measured results, and thus white noise is commonly used in theoretical calculations. One additional property of white noise is that two white noise samples are statistically independent (and hence uncorrelated) no matter how closely spaced they are in time. Of course, one cannot take actual samples of our mathematical fiction. In real life, all measurements are made using finite-bandwidth instruments (say $W$ Hz), and so the noise samples that we can measure are those obtained after some implicit filtering of the white noise that we set out to sample. In particular, noise samples less than $W^{-1}$ seconds apart definitely are correlated. Noise samples further apart in time also are correlated but the correlation values are small enough that it is reasonable to treat them as negligible and assume that the samples are indeed independent and uncorrelated. For more on this viewpoint, read Appendix A of this lecture note

If a continuous-time noise process is sampled at the Nyquist rate and converted to a discrete-time sequence of samples, then each sample can be taken to be a random variable (usually zero-mean Gaussian) independent of all other samples. Thus, a discrete-time white noise process is a sequence of independent (and hence uncorrelated) identically distributed zero-mean random variables. If the random variables are Gaussian (as is almost always assumed), the process is called a discrete-time white Gaussian noise process. In any case, it is not necessary to say uncorrelated white noise: white noise is always uncorrelated.

When 2 signals are said to be correlated, it means that their correlation coefficient is non zero. The correlation coefficient is a value between -1 and +1, that depends on how the 2 signals vary together. If they vary largely "independently", then the correlation is close to 0 and the signals are said to be uncorrelated. If the correlation coefficient is close to 1, they are strongly correlated and if it is close to -1, they are strongly anti-correlated.

Auto correlation of a signal is a series that shows patterns within a signal. Each point of this series is the correlation coefficient of the signal with a delayed (or advanced) version of itself.

Uncorrelated noise refers to noise that has a zero autocorrelation function. So, every point in the noise signal is "independent" of every other point. So, even if you have signal values for large time epochs, you cannot predict the next value.

"Whiteness" of a noise refers to the flatness of its power spectrum. It is possible for uncorrelated noise to not be white, but pink(!) or other colors based on the power spectrum.

So, uncorrelated white noise is noise that is both uncorrelated and has a flat power spectrum. White Gaussian noise is an example of uncorrelated white noise.

• IMO, The Auto-correlation of white noise tends to an impulse, not to a uniformly zero function. Please correct this in your answer. This is by virtue of Wiener-Khinchin Theorem which states that the autocorrelation function of a wide-sense-stationary random process has a spectral decomposition given by the power spectrum of that process. Jun 6 '18 at 8:15
• The original question was about correlation with an example of uncorrelated white noise. So, the answer was simply about what is correlated vs uncorrelated, and the meaning of the term "white noise". The auto correlation of white noise was not the subject of this question, IMHO. Aug 8 '18 at 7:49

As Steven explained, in statistics 2 events are correlated if knowing the outcome of one gives information to predict the outcome of the other one.

For instance, if you throw a coin twice, the statistics say that the two events are independent, and knowing one won't affect the prediction on the other one. But if you have a cards deck, and you pick the ace of spades (without putting it back), you know that's impossible that the next times it will come out again. The events are dependent.

Correlation is somewhat similar: if your wife starts taking sewing lessons at 11 pm twice a week, and at the same time your best friend is in business meetings, you may think that the two events share some properties.

A stochastic process describes the behavior of a stochastic event over time. It means that you can have many different values at any time, and any possible outcome is defined as a function of time. The theory is complicated, but think to it as an immense music library. At any instant, one song of the library will be playing, and you can generate infinite playlists. (sorry for the lame example)

In this system, you can have two types of correlations: in time and in state. The time correlation says that knowing what's played at a certain time, you can predict (to a certain extent) what will be played in a few seconds. The state correlation says that from the same knowledge (what's being played now) you can estimate what else could have been played at the same time (maybe it was set for playing rock music at 5 pm).

Electronic noise is a very broad term that indicates everything that mixes together with your signal without giving any useful information, and making the useful part less clear. In communications, there is a lot of effort in getting the information to the other side, and this implies making the signal to stand out in the noise. It can be done increasing the power of the signal in transmission, shielding the communication medium, filtering or in other ways.

Since noise can be due to different phenomena, it will also have different properties. Thermal noise is due to the vibration of charge carriers in conductors, so you can expect it to depend on the temperature of the same; interference happens when another signal generator (think to a microwave oven) transmits over your signal. In this last case, if you know what the transmitter is doing, you can counter the effect in a more directed way (for instance, a band-stop filter centered at the exact frequency).

So knowing the statistic properties of signal and noise can help in separating the former from the latter, when analysis is necessary.