# Lossless FFT on audio data

I have an application where I'll be receiving an audio stream, much like using headphones, into my processor. I would like to run FFT and various DSP algorithms on the sampled audio and then spit the audio back out as if the device wasn't there. Basically doing DSP in real-time in between samples I guess. The processor I'm using is the MKL26Z128xxx4, mounted on the FRDM-KL26Z development board. It has a single ARM Cortex M0+ core and is programmed in C/C++ and can use the THUMB Assembly instruction set.

How would I do this without losing any of the streamed audio?

• You'd have an easier time with a Cortex M4, which can do a multiply and add in one clock tick Apr 12, 2014 at 1:10
• Thanks! Right now this is just in the development process so I guess, say I didn't have that right now. Apr 12, 2014 at 1:12
• I'd respond by recommending the STM32F4 Discovery, for \$15 and almost no change to your toolchain. You don't say what DSP you want to do, but it will be easier on a DSP than on an M0 Apr 12, 2014 at 1:23
• Hmm, okay. I mainly want to accomplish an FFT, ideally on as many samples as possible without loss of data integrity. I guess I'm more concerned with the concept of doing this on a single-core controller. Multicore is simple since you just get data, copy it to another core and delay the output of it while the other core is doing the processing algorithms Apr 12, 2014 at 1:25
• Is the audio signal raw or preprocessed? Apr 12, 2014 at 13:26

FFT needs to sample at at least twice the frequency of your signal, and should not introduce any loss. Loss is introduced on compression or encoding. Your input and outputs should be able to match the frequency, amplitude, and frequencies of the signal you are sampling.

Your biggest challenge is processing the DSP algorithms in real time, which depends on the algorithms you use, how many, and how your processor can keep up with it, with minimal delay. Like fractions of a second, to stay in sync with anything else (Video, other audio, or live instruments).

You need to do some calculations on what you need, through testing, before you can find out if your platform is good enough, or where you need to rethink or cut around.

• Thanks! Think you could point me in the proper direction for those calculations? I'm not sure which ones I would need to complete. Apr 12, 2014 at 1:34
• Try demoing your processing offline first, using Matlab or one of the free alternatives. Then figure out how much power you need to do the same thing in realtime. Apr 12, 2014 at 15:48

Unfortunately, you will have to accept losses in your signal if you try to reconstruct them from the samples. Unless you make certain strong assumptions about the audio signal the Nyquist-Shannon sampling theorem doesn't help. A general raw audio signal will not satisfy these conditions.

If you take $K$ samples at a rate of $\frac{1}{2B}$ starting at $1/2B$ where $B$ is the bandwidth then the functions $$\frac{\sin(2\pi t B)}{2 t B}$$ and $$\frac{\sin(2\pi t B - 2(K+1) \pi)}{2 t B - 2(K+1)}$$

are indistinguishable from each other using only the samples taken at times $\frac{1}{2B}, \frac{2}{2B},...,\frac{K}{2B}$ as both functions evaluate to zero at these points. The exact reproduction of a signal from Nyquist-Shannon sampling theorem assumes an infinite number of sample points.

Things get slightly worse once FFT is involved. FFT reproduces the signal as a finite sum by using a finite set of frequencies determined by your sampling rate. If you have a raw audio signal, you have an possibly infinite set of frequencies being received despite being limited by $B$.

In more mathematically correct language the space of functions with bandwidth $\leq B$ is a infinite dimensional vector space. Once you fix a sampling rate the vector space of functions constructed via FFT is finite dimensional and thus not all signals can be recovered exactly via FFT.

Any digital sampling involves low-pass filtering (and/or potential aliasing) and quantization. Any ADC or DAC in the path will add delay, both due to buffering, potential processing (noise filtering or successive approximation), as well as physical and electrical time constants. Thus you will have to specify some finite thresholds in bandwidth, noise floor and delay, as equal to your "no loss" criteria, or you have an impossibility using any processor chip.

Whether or not you do an FFT for analysis is irrelevant if you bypass stream your data directly to the output as well as sending it to the FFT.

An FFT is a block-based process, so it isn't done "between samples" but on entire blocks, buffers or arrays of samples, thus adding latency to any FFT analysis (e.g. waiting for a big enough buffer to fill before starting).

An FFT of length N has to be done using processor arithmetic (accumulation registers) that is at least log2(N) bits larger than the bit size of your samples, or the FFT itself will introduce (additional) truncation, rounding or clipping errors.