Lossless FFT on audio data

Question

I have an application where I'll be receiving an audio stream, much like using headphones, into my processor. I would like to run FFT and various DSP algorithms on the sampled audio and then spit the audio back out as if the device wasn't there. Basically doing DSP in real-time in between samples I guess. The processor I'm using is the MKL26Z128xxx4, mounted on the FRDM-KL26Z development board. It has a single ARM Cortex M0+ core and is programmed in C/C++ and can use the THUMB Assembly instruction set.

How would I do this without losing any of the streamed audio?

Answer 1

FFT needs to sample at at least twice the frequency of your signal, and should not introduce any loss. Loss is introduced on compression or encoding. Your input and outputs should be able to match the frequency, amplitude, and frequencies of the signal you are sampling.

Your biggest challenge is processing the DSP algorithms in real time, which depends on the algorithms you use, how many, and how your processor can keep up with it, with minimal delay. Like fractions of a second, to stay in sync with anything else (Video, other audio, or live instruments).

You need to do some calculations on what you need, through testing, before you can find out if your platform is good enough, or where you need to rethink or cut around.

Answer 2

Unfortunately, you will have to accept losses in your signal if you try to reconstruct them from the samples. Unless you make certain strong assumptions about the audio signal the Nyquist-Shannon sampling theorem doesn't help. A general raw audio signal will not satisfy these conditions.

If you take \$K\$ samples at a rate of \$\frac{1}{2B}\$ starting at \$1/2B\$ where \$B\$ is the bandwidth then the functions $$ \frac{\sin(2\pi t B)}{2 t B}$$ and $$ \frac{\sin(2\pi t B - 2(K+1) \pi)}{2 t B - 2(K+1)}$$

are indistinguishable from each other using only the samples taken at times \$\frac{1}{2B}, \frac{2}{2B},...,\frac{K}{2B}\$ as both functions evaluate to zero at these points. The exact reproduction of a signal from Nyquist-Shannon sampling theorem assumes an infinite number of sample points.

Things get slightly worse once FFT is involved. FFT reproduces the signal as a finite sum by using a finite set of frequencies determined by your sampling rate. If you have a raw audio signal, you have an possibly infinite set of frequencies being received despite being limited by \$B\$.

In more mathematically correct language the space of functions with bandwidth \$\leq B\$ is a infinite dimensional vector space. Once you fix a sampling rate the vector space of functions constructed via FFT is finite dimensional and thus not all signals can be recovered exactly via FFT.

Answer 3

Any digital sampling involves low-pass filtering (and/or potential aliasing) and quantization. Any ADC or DAC in the path will add delay, both due to buffering, potential processing (noise filtering or successive approximation), as well as physical and electrical time constants. Thus you will have to specify some finite thresholds in bandwidth, noise floor and delay, as equal to your "no loss" criteria, or you have an impossibility using any processor chip.

Whether or not you do an FFT for analysis is irrelevant if you bypass stream your data directly to the output as well as sending it to the FFT.

An FFT is a block-based process, so it isn't done "between samples" but on entire blocks, buffers or arrays of samples, thus adding latency to any FFT analysis (e.g. waiting for a big enough buffer to fill before starting).

An FFT of length N has to be done using processor arithmetic (accumulation registers) that is at least log2(N) bits larger than the bit size of your samples, or the FFT itself will introduce (additional) truncation, rounding or clipping errors.