16
\$\begingroup\$

Is the following statement from a book wrong?

enter image description here

I thought sampling with twice the signal's highest frequency component would be adequate to completely recover the signal. But above it says sampling twice creates a saw-tooth like wave. Is the book wrong?

\$\endgroup\$
  • 14
    \$\begingroup\$ To completely recover the signal is they key phrase. Nyquist doesn't say that you can connect the samples with a straight line and get the original signal, but that the information needed to recover the signal is there. So the book is right about what the signal looks like when you connect the dots, and Nyquist is right about what you are able to recover from the samples. \$\endgroup\$ – John D May 3 at 14:26
  • 12
    \$\begingroup\$ Technically, a sampling frequency of exactly 2x the input does not allow for reconstructing the original signal, unless you somehow also know that you are sampling at the peaks/troughs. Just a little bit faster is needed in theory (and significantly faster in practice). \$\endgroup\$ – Reinstate Monica May 3 at 14:30
  • 7
    \$\begingroup\$ Nyquist specifically mentions that the signal is band limited. Band limiting the input is often mentioned but band limiting the output is not often mentioned. If you band limit the triangle wave to your original signal you will get a sine wave back. \$\endgroup\$ – vini_i May 3 at 14:39
  • 7
    \$\begingroup\$ If you count the little dots that denote sampling, the sampling rate is off by a factor of two in each of those diagrams -- the sampling is at 2x, 4x, and 8x, respectively. \$\endgroup\$ – TimWescott May 3 at 14:55
  • 4
    \$\begingroup\$ @sidA30 The correct procedure is to wait until you have time to write an answer, not just break the policy at your convenience. \$\endgroup\$ – pipe May 3 at 15:13
16
\$\begingroup\$

I thought sampling with twice the signal's highest frequency component would be adequate to completely recover the signal. But above it says sampling twice creates a saw-tooth like wave. Is the book wrong?

The book is wrong, but not for the reason you think. If you squint at the dots that indicate samples, it's sampling at twice the frequency that it says.

So first, you should draw some signals and sample them yourself (or use a math package, if you're not up to pencil and paper).

Second, The Nyquist theorem says that it is theoretically possible to reconstruct a signal if you already know that the signal content's spectrum is strictly less than 1/2 the sampling rate.

You reconstruct the signal by low-pass filtering it. Before filtering, the signal can be distorted, so you have to know what you're looking at to see that the result might look OK. Furthermore, the closer your signal content's spectrum is to the Nyquist limit, the sharper the cutoff needs to be in your anti-alias and reconstruction filters. This is fine in theory, but in practice a filter's response in the time domain gets longer roughly in proportion to how sharply it transitions from its passband to its stopband. So in general, if you can, you sample well above Nyquist.

Here's a picture that goes with what your book should have said.

Case A: one sample per cycle (samples made obvious)

Case B: two samples per cycle, landing on the intersections -- note that this is the same output as the one sample per cycle case, but only because I sampled the first one at the intersections.

Case C: Again, two samples per cycle, but this time at the extremes. If you sample at exactly twice the signal component frequency, then you can't reconstruct. In theory you could sample oh-so-slightly lower, but you'd need a filter with an impulse response that spans enough of the result so that you can reconstruct.

Case D: Sampling at 4x the signal frequency. If you connect the dots you get a triangle wave, but it's not correct to do so -- in sampled time, the samples only exist "at the dots". Note that if you put this through a decent reconstruction filter you'll get a sine wave back, and if you change the phase of your sampling then the output will be shifted equally in phase, but its amplitude won't change.

corrected sampling

\$\endgroup\$
  • 1
    \$\begingroup\$ @ThePhoton I count 2, 4, 8 samples per cycle. \$\endgroup\$ – jpa May 3 at 16:29
  • 2
    \$\begingroup\$ As an addendum to what Nyquist's theorem theoretically says, it also assumes you have an infinitely long period of data to sample. In the "sawtooth" example, the book cheats by drawing a signal that has higher frequency components. If you had an infinite series of those high/low/high/low patterns the only signal which can generate forever without a higher frequency component will be the sine wave. \$\endgroup\$ – Cort Ammon - Reinstate Monica May 3 at 16:38
  • 2
    \$\begingroup\$ Did you mean that we should sample above Nyquist? \$\endgroup\$ – Ronan Paixão May 3 at 17:35
  • 1
    \$\begingroup\$ @ThePhoton If you look more closely at the source signal, you can see very faint dots, this makes it 2 4 8, you might have to get further or closer to your monitor to see all the dots in the last signal \$\endgroup\$ – Ferrybig May 3 at 17:53
  • 5
    \$\begingroup\$ @RonanPaixão Yes, in general you want to sample above the Nyquist rate. There's a tradeoff -- sampling fast is expensive in terms of acquisition hardware and memory storage, but as you sample more slowly the necessary anti-aliasing and reconstruction filters get more expensive. So you dicker and think and do big spreadsheets and then you decide -- and five years later on technology has advanced to the point that your "best" solution looks hopelessly wrong. \$\endgroup\$ – TimWescott May 3 at 17:59
5
\$\begingroup\$

Picture B is extremely wrong. It contains very sharp corners in the output signal. Very sharp corners equals very high frequencies, a lot higher then the sample frequency.

In order to fulfill the Nyquist sample theorems, you need to low pass filter the reconstructed signal. After low pass filtering the signal B would look like the input signal, not like like a triangle (as all the sharp corners can not pass the low pass filter).

To be exact you need to lowpass both the input signal and the output signal. The input signal needs to be low pass filtered to max half the sample frequency in order to not "fold" higher frequencies.

Sadly, it is a common misrepresentation of how sampling works. A more correct description will use the sinc function for reconstruction (I recommend a search for sinc function).

In real world applications it is impossible to have a "perfect" low pass filter (passing all frequencies below and blocking all above). This means that you would normally sample with a frequency at least 2.2 times the max frequency you want to reproduce (example: CD quality sampled at 44.1 kHz in order to allow a 20kHz max freqency). Even this difference would make it hard to create analog filters -- most real world applications "oversample" as does the low pass filter partly in the digital area.

\$\endgroup\$
  • 4
    \$\begingroup\$ To be fair, you are interpreting the graphs in a different way than they are presented - there is no claim that they are a "reconstruction" only that they are the digitized output of the ADC. Connecting the dots with lines is a common temptation and feature of systems that minimally represent data without trying to interpret it. \$\endgroup\$ – Chris Stratton May 3 at 20:07
  • 1
    \$\begingroup\$ I agree to the temptation. Often though I see it depicted as steps instead, most software programs shows stairs when zoomed in close. The problem is when people start interpreting the lines (or stairs) as the true meaning of the sampled signal. Most often the samples will be replayed later. \$\endgroup\$ – ghellquist May 4 at 6:43
  • \$\begingroup\$ The question clearly shows monofrequency input. Aliasing is not the point of the question. \$\endgroup\$ – Scott Seidman May 5 at 11:01
3
\$\begingroup\$

The sampling theorem states that the signal can be perfectly reconstructed if the sampling frequency is strictly greater than the highest frequency contents in the signal. But that reconstruction is based on inserting (infinite) sinc pulses at each sample. From a theoretical point of view this is a very important result, but in practice impossible to achieve exactly. What is described in the book page is a reconstruction method based on drawing straight lines between the samples, which is something completely different. So, I would say the book is correct, but it does not have anything to do with the sampling theorem.

\$\endgroup\$
  • 4
    \$\begingroup\$ Not quite, "strictly greater then twice the BANDWIDTH" is the usual formulation, and the difference matters (This is why subsampling as a means to downconvert RF works). \$\endgroup\$ – Dan Mills May 3 at 20:09
  • \$\begingroup\$ Yes, but to explain the sampling theorem with respect to the question I would still go for highest frequency. The question is about sampling a pure sine, and then it can be confusing to introduce the bandwidth. \$\endgroup\$ – StefanH May 4 at 8:14
3
\$\begingroup\$

A very nice overview paper is Unser: Sampling - 50 years after Shannon. Your problem arises from the fact that pure, infinite sine signals are not covered by the Shannon sampling theorem. The applicable theorem for periodic signals is the earlier Nyquist sampling theorem.


The Shannon sampling theorem applies to functions that can be represented as

\$ x(t)=\int\limits_{-W}^WX(f)e^{i2\pi ft}\,df \$

where X is a square-integrable function. Then this signal can be exactly represented from discrete samples as

\$ x(t)=\sum\limits_{k=-\infty}^\infty x(k\frac{T}2)\frac{\sin(\pi W(t-k\frac{T}2))}{\pi W(t-k\frac{T}2)} \$

with \$T=\frac1{W}\$ a "period". Note that the perfect reconstruction depends on samples from arbitrarily large times in the future and past. As their influence only falls as \$\frac1t\$, truncating the sum has to include a rather large number of terms to reduce errors.

A pure sine function is not contained in that class, as its Fourier transform is composed of Dirac-delta distributions.


The earlier Nyquist sampling theorem states (or re-interprets an earlier insight) that if the signal is periodic with period T and highest frequency W=N/T, then it is a trigonometric polynomial

\$ x(t)=\sum\limits_{n=-N}^NX_ne^{i2\pi\frac{n}{T}t} \$

with 2N+1 (non-trivial) coefficients and these coefficients can be reconstructed (by linear algebra) from 2N+1 samples in the period.

The case of a pure sine function falls in this class. It promises perfect reconstruction if 2N+1 samples over a time NT are taken.

\$\endgroup\$
3
\$\begingroup\$

What has been shared from the book does not say anything about "Nyquist Sampling Criterion" -- it is only talking about point-sampling a sine wave with a hypothetical ADC, and then (implicitly) constructing an output signal using a (not mentioned) simple DAC that performs a linear interpolation between the sample values.

Given that context, the thesis statement of 'FIGURE 6.10' is generally correct and well demonstrated.

As the sampling frequency of the ADC is increased, the fidelity of the digitized signal improves.

If you wanted to talk about the fidelity of an idealized reconstruction, that's a different matter entirely. Any discussion of the Nyquist rate, implies the use of sinc interpolation which, again, isn't mentioned in the figure shown.


The real flaw in this figure is the idea that a point-sample is a meaningful concept in engineering. Practically speaking, an ADC will be connected to a sensor component that works by accumulating a real-world input signal over some period of time.


It's funny, however, that figure is apparently wrong (off by a factor of two) about the specific sampling frequencies shown in the diagrams -- although the "Output" shown is only affected by this in case 'C'.


Using the quoted statement above, I found an eerily similar diagram in "A Practical Approach to Neurophysiologic Intraoperative Monitoring" in a discussion about EEG waveform processing. For what it's worth, that discussion includes the following:

The theorem describing the minimum sampling frequency required for an ADC to faithfully represent an analog signal is known as the Nyquist theorem. It states that the sampling frequency of an ADC must be greater than twice that of the fastest frequency component of a waveform.

\$\endgroup\$
  • \$\begingroup\$ ...some period of time and/or space -- when translating physical phenomena to digital samples. Crudely, there will always be an inherent low-pass filter. \$\endgroup\$ – nobar May 4 at 20:31
  • \$\begingroup\$ Something I just came across that address the inherent low-pass filter: engadget.com/2019/05/04/… \$\endgroup\$ – nobar May 6 at 1:52
  • \$\begingroup\$ A point that I'm getting at is that a perfect reconstruction of a physical signal is basically impossible (in the general case), and that a best-possible reconstruction should take into account the effective lowpass filtering that is inherent in the physical to digital conversion. \$\endgroup\$ – nobar May 6 at 1:55
  • \$\begingroup\$ This video (shared in the question comments) loses some credibility @8:17 when he says that 2D image pixels are "conceptually, infinitely small points". It ignores a lot of detail about how image samples are actually captured -- and what information they represent. \$\endgroup\$ – nobar May 9 at 13:30
  • \$\begingroup\$ ...Although it is true that digital pixel samples are captured and stored as values that are discrete in their representation of time/space -- that doesn't mean that they are "infinitely small points". \$\endgroup\$ – nobar May 9 at 14:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.