5
\$\begingroup\$

I am trying to implement an algorithm on audio signals. The algo operates on the signal on a frame-by-frame basis. One of the intermediate stages requires addition of the signal to the previous frames. The output frame is saturated at the end of this stage.

Clarification: The output frame of the addition operation is in double, though the final output frame HAS to be in short. Thus the problem with saturation.

I have tried the following for prevention of saturation:
1. hard-limiting: Limit any sample to a pre-defined max_val.
2. Normaliation: Compute the max. of the frame and then scale every sample in the frame appropriately.

None of these approaches have worked since I have to work on a frame by frame basis.

Any help on preventing saturation of signals is most welcome.

\$\endgroup\$
  • \$\begingroup\$ What is physically/virtually saturating? What is the maximum/minimum of the input signal? Is the entire signal saturated, or is it only the peaks? \$\endgroup\$ – W5VO Nov 29 '10 at 8:16
  • \$\begingroup\$ the values of the signal frames (post addition operation) saturate. the output of the above operation is double, but has to be made into short. hence, the problem with saturation. \$\endgroup\$ – Sriram Nov 29 '10 at 13:40
  • \$\begingroup\$ Is this for music or voice? \$\endgroup\$ – Kellenjb Nov 29 '10 at 17:43
  • \$\begingroup\$ @Kellenjb: for both. \$\endgroup\$ – Sriram Nov 30 '10 at 7:03
8
\$\begingroup\$

Your Attempts

By definition when you "hard-limit" values in your code you are causing saturation. It may not be saturation in the sense of overflowing your short, but you are still distorting the wave when it goes over a certain point. Here's an example:

Saturation

I realize you probably aren't hard-limiting on the bottom, but I had already drawn it before I realized that.

So, in other words, the hard limiting method won't work.

Now for your second approach, this method will cause you to do what some audio people actually do intentionally. You are causing every frame to be as loud as it possibly can. This method can work OK if you get the scaling right and are fine with your music sounding loud all of the time, but it isn't great for most people.

One Solution

If you know the max possible effective gain that your system can create, you can divide your input by this much. To figure out what this would be you will need to step through your code and determine what the max input is, give it a gain of x, figure out what the max output is in terms of x, and then determine what x should be in order to not ever saturate. You would apply this gain to your incoming audio signal before you do anything else to it.

This solution is OK, but isn't great for everyone as your dynamic range can be hurt a little since you usually wont be running at max input all of the time.

The other solution is to do some auto-gaining. This method is similar to the previous method, but your gain will change over time. To do this you can check your max value of each frame of your input. You will use will store this number and place a simple low pass filter on your max values and decide what gain to apply with this value.

Here is an example of what your gain versus input volume would be:

auto-gain

This type of system will cause most of your audio to have a high dynamic range, but as you start getting close to the max volume you can have you slowly reduce your gain.

Data Analysis

If you are wanting to find out what type of values your system is actually getting in real time then you will need to have some type of debugging output. This output will change depending on what platform your running on, but here's a general gist of what you would do. If you are on an embedded environment you will need to have some serial output. What you will do is at certain stages in your code output to a file or screen or something you can grab the data from. Take this data and put it in excel of matlab and graph all of them versus time. You will probably very easily be able to tell where stuff is going wrong.

Very Simple Method

Are you saturating your double? It doesn't sound like it, instead it sounds like you are saturating when you switch to a short. A very simple and "dirty" way of doing this is to convert the max of your double (this value is different depending on your platform) and scale that to be the max value of your short. This will guaranty that assuming you don't overflow your double that you wont overflow your short either. Most likely this will result in your output being much softer then your input. You will just need to play around and use some of the data analysis that I described above to make the system work perfectly for you.

More Advanced Methods that probably don't apply to you

In the digital world there is a trade off between resolution and dynamic range. What this means is that you have a fixed number of bits given to your audio. If you decrease the range that your audio can be in then you increase the bits per range that you have. If you think about this in the sense of volts and you have 0-5v input and 10bit adc then you have 10bits to give to a 5v range, usually this is done linearly. So 0b0000000000 = 0v, 0b1111111111 = 5v and you linearly assign the voltages to the bits. In reality, with audio, this isn't always a good use of your bits.

In the case of voice, your voltages versus probability of those voltages look something like this:

pdf

This means that you have a lot more of your voice in the lower amplitude and just a little amount in the high amount. So instead of assigning your bits lineally, you can remap your bits to have more steps in the lower amplitude range and thus less in the upper amplitude range. This gives you the best of both worlds, resolution where most of your audio is at, but limit your saturation by increasing your dynamic range.

Now, this remapping will change how your filters act and will probably need to rework your filters, but that is why this is in the "advanced" section. Also, since you are doing your work with a double and then converting it to a short, your short will probably need to be linear anyways. Your double already gives you much more precision then what your short will give you so there is probably no need for this method.

\$\endgroup\$
  • \$\begingroup\$ he specified that he does not know the signal before he is receiving it and is in a real time environment. he also specified he was keeping the same hard limit for the entire signal, which is where the saturation comes from. \$\endgroup\$ – Kortuk Nov 29 '10 at 17:43
  • \$\begingroup\$ +1 auto-gaining / basic signal division by max gain is what i would probably do. No need to know the input signal ahead of time for that. \$\endgroup\$ – Mark Nov 29 '10 at 19:04
  • \$\begingroup\$ @Mark, @kellenjb, now that I see that he has absolutely no control over your input, and you are not getting input, i would suggest auto-gain. I am not sure this is another method. I would make the gain level determined slowly over a long time frame. \$\endgroup\$ – Kortuk Dec 1 '10 at 14:43
2
\$\begingroup\$

If this is at all similar to how values overflow in C, by the time you have done the addition, you would have overflowed. So you would need to halve the current frame and the previous frame value that you are adding before you add, then add, then optionally normalise.

But in any case, you will have a hard time keeping a consistent level (audio compression) if you have only one frame to work on at a time, because you need a long term average that doesn't jump around.

\$\endgroup\$
  • \$\begingroup\$ yes. that is my problem. i dont have any long-term statistics (not in a real-time environment anyway). is there some technique in the literature that gives a way to do this? right now, i just choose a number at random (very high) and scalar multiply with the signal. that takes care of the problem but fares miserably when a signal with different stats is used. please also read the previous comment (above) for some more details about the problem. \$\endgroup\$ – Sriram Nov 29 '10 at 13:43
1
\$\begingroup\$

I take it you are having to carry over the result because you are doing real time dsp with audio.

I have seen this issue before when students make an error in how they carry over the result of the idft. Often they incorrectly carry things over and end up causing a continuous buildup. Based on your comment to martin you seem to have avoided this.

Second, This is why the radio has a volume knob you have to use. When you change radio stations from a transmitter nearby to one that is far away you have to change what your volume level is to hear it at the same volume. I would suggest you add a user controllable gain knob, then if it saturates it is in their control to change it. A user will normally do this automatically as it gets louder as they do not like it. They will also turn it up when needed.

This knob would change what you are scaling your values by. I hope this helps, with more information about your application I could give other help. You could create a very slow response average(Low Low pass filter) that takes many frames to change significantly and use it to give gain control. This means a sudden spike will cause saturation, but if your audio volume changes slowly you will easily compensate for it. Even if your audio changes quickly, this type of filter will save the user turning it down and up if each relative volume level is sustained for an extended period.

\$\endgroup\$
  • \$\begingroup\$ yes, real-time or at least simulated real-time dsp with audio is what i am trying to do. unfortunately, i am not sure if i can provide any more details :(. i am interested in knowing a little more about the slow response average that you mention. what about automatic gain control? \$\endgroup\$ – Sriram Dec 1 '10 at 6:48
1
\$\begingroup\$

The only way to do this is to ensure that you can't saturate. Your algorithm presumably has some kind of "maximum-gain" which means you know the most the signal can grow per frame. You have to ensure that the first frame has enough headroom for the signal not to saturate by the end.

Your example has an addition, which (unless you know more than you've said so far about it) grows your output by at most one bit over the input (with more knowledge about the system, you may be able to say it will average no more than eg. 2.4 bits over the whole set of frames). But in this simple case, if you have 8 frames, you'll have to leave 8-bits of headroom over the original samples. So if you sample at 8 bits and calculate in 16 bits, you should be OK.

\$\endgroup\$
  • \$\begingroup\$ But what if my output has again got to be in 8 bits?? that is my problem. it is not the saturation of the data type used for addition that is the problem, it is the constraints imposed by the size of the output that is. \$\endgroup\$ – Sriram Dec 1 '10 at 6:02
1
\$\begingroup\$

@Kellenjb , thanks for the images -- I think that helped me understand things greatly.

@Sriram , I wish you would say a few more words as to what you wish would happen.

I'm guessing that what you wish would happen is something like "I want an output that is always as loud as possible -- right on the verge of clipping. But I don't want the volume to radically change from one frame to the next. I want the volume to change more like a human slowly increasing the volume knob during the quiet parts, and somewhat more quickly turning down the volume when it starts to clip. But I don't want an actual human to turn the volume knob -- I want the software to automatically adjust the volume for me."

I'm also assuming that your system needs to spit out a frame of data within a frame or two of getting in a frame of data, in real time -- so I'm not allowed to analyze the entire song, and then afterward pick the perfect volume level and hold it constant throughout the entire song.

So I guess you want to set up a pipeline something like this:

(1) One interrupt routine collects data into a buffer. Often people set up 2 frames worth of storage: This routine slowly fills up one frame while the next stages are working with data in the other frame. Then when the one frame gets full you ping-ping double-buffer switch to filling up the other frame, while the next stages switch to working with the one frame. Some people doing algorithms such as finite (FIR) filters buffer many, many frames worth of storage here.

(2) In the "background" main loop, the software runs some DSP algorithm on the latest "full" input buffer and generate some output results in the output buffer. Often people set up 2 frames worth of output buffer storage, so this routine quickly fills up one output frame, while the next stages slowly drain data from the other frame, with a ping-pong double-buffer scheme. Some people doing algorithms such as infinite (IIR) filters buffer many, many frames worth of storage with these output values. Most of the time, the DSP algorithm takes as input the previous outputs of this stage, not the outputs of the next stages. This stage is where most people run into problems with clipping and saturation, so most of the answers I see here focus on this step. But from your responses, it sounds like you've already figured out that you need a wide data type with many "extra" bits of precision to calculate these intermediate values.

(3) In the main loop, the software multiplies each value of the intermediate frame by some scaling factor V to generate a "scaled value". (It does not store this value in any buffer).

(4) In the main loop, the software reduces each "scaled value" to the final output type (8 bits?) required by your output device, and store the result in yet another buffer. (The output of this stage should almost certainly not be fed back into the DSP algorithm in step 2. "The BTC Sound Encoder" by Roman Black is one of the very few algorithms I know of where such feedback is useful).

(5) One interrupt routine takes one sample at a time from the output buffer from (4), and spits it out to the output device. (This may also use a double-buffered ping-pong arrangement). Often the input sample rate and the output sample rate are the same (44100 Samples/s), and the same interrupt routine does (5) and (1).

From your response to Martin, I get the impression that one particular fixed volume setting V1 for V may be adequate for one song, and some other fixed volume setting V2 may be adequate for some other song. Imagine that you could listen to the whole song, and calculate a perfect volume for that song -- how would you do that? Is it OK if a few samples of the song get clipped?

Perhaps something like this relatively simple automatic gain control (AGC or VOGAD) would solve your problem:

  • Every time stage 2 finishes producing a frame, calculate a new V value that will be used to scale the samples in that frame during step 3. One of the simplest ways: find the most positive (maximum) value and most negative value in that frame, and multiply both values by the current V that was used for the previous frame. Check if either sample will clip (i.e., the result is above +127 or below -127) or are close to clipping (i.e., above +63 or below -63), and set the volume multiplier VM appropriately:

    • If the volume is too low (none of the samples clip, none of the samples are anywhere close to clipping), set the volume multiplier to some number slightly greater than 1: VM = VM_UP, where VM_UP is something like 1.01 or 1 + 2^(-12).

    • If the volume is too high (some of the samples clip, or too many of the samples are close too clipping, or both), set the volume multiplier VM to some number slightly less than 1: VM = VM_DOWN, where VM_DOWN is something like 0.98 or 1 - 2^(-8).

    • If the volume looks OK (not to high, not too low) keep VM = 1.

  • Periodically (ideally once every sample, but if that eats to much time once per frame or once per millisecond or once every tenth of a second may be adequate), re-adjust the volume with a saturating multiply:

    • V := (V * VM).
    • If V is now too small (less than V_min), set V := V_min.
    • If V is now too big (more than V_max), set V := V_max.

You'll want to tweak the compile-time constants VM_UP and VM_DOWN until it subjectively "sounds right". Perhaps set VM_DOWN so that, with very loud inputs, it takes a second or two to crank the volume knob from the maximum volume to the minimum volume. Perhaps set VM_UP so that, with absolute silence, it takes ten seconds or so to very slowly crank the volume knob from the minimum volume to the maximum volume.

You could guarantee that you'll never saturate by instantly reducing the volume V whenever a frame looks like it might saturate, such that (with the reduced volume V) it just barely hits the peaks of +127 or -127. But I suspect that it will sound better to slowly reduce the volume, and allow a few samples to saturate for a few frames.

There are a wide variety of more complicated ways to (a) envelope/peak/"perceptual loudness" detection: calculate a number that indicates whether a frame of data is too loud or too soft, and by how much. There are also a wide variety of more complicated ways for an AGC to (b) use that number from the current frame (and perhaps for the past few frames) and the volume used during the last frame in order to calculate the volume to use for the next frame.

Further reading:

\$\endgroup\$
0
\$\begingroup\$

You can use Linear predictive coding to compress the signal down into a "residual" and a filter to recreate that frame of audio later on. It's pretty complicated, but here is a good paper. If you keep the residual exactly, this is a lossless algorithm. You can also just use fewer bits to encode the residual and keep good quality.

J. Makhoul. Linear prediction: A tutorial review. Proceedings of the IEEE, 63(4):561–580, April 1975.

And a chapter in DAFX has MATLAB algorithms written for you.

U. Zolzer, editor. DAFX - Digital Audio Effects. John Wiley & Sons, Ltd, Baffins Lane, Chichester, West Sussex, PO 19 1UD, England, 2002.

Basically given an input signal frame you can compute a spectral envelope and a residual. See the pictures below, and note the amplitude of the residual. alt text alt text

Use the filter from the second figure and give it the residual when you want to recreate the sound. Notice that the residual has a fairly stable amplitude, and note the units. You need to be able to encode at least that. The filter is an "LTI" filter and has nice properties you could probably use to combine frames.

\$\endgroup\$
  • \$\begingroup\$ Is there any public link to this? \$\endgroup\$ – Kellenjb Nov 29 '10 at 17:40
  • \$\begingroup\$ I read this, and I am sorry to say I understood most of it. I do not understand how it fixes anything. It seems you are saying if you give a greater delay you can compensate for the volume better. \$\endgroup\$ – Kortuk Nov 29 '10 at 17:40
  • \$\begingroup\$ No pubic link, sorry, but I know the DAFX book is available online somewhere. (google for an ebook) I'm not adding delays, except for the delays to implement the filter in the second figure, which is used to recreate the sound at a later time. This only deals with encoding. It transforms an input signal into a filter and a residual. The residual has a tiny amplitude and can be encoded with a small number of bits. Filters can be added by using them in series. The residuals can be added without overflow since they have small values, and stored as such as an intermediate step before the final out \$\endgroup\$ – Chris H Nov 29 '10 at 18:13
  • \$\begingroup\$ I am not sure how LPC would solve the problem. even if adding the residuals does not overflow, convolving it with the filter to obtain the original frame and then having to cast it into a data type with shorter range would cause saturation. this algo must happen in real-time (or at least simulated real-time), so my time limit for processing is also very low. \$\endgroup\$ – Sriram Dec 1 '10 at 6:37
  • \$\begingroup\$ Obviously if you're adding huge numbers you're not going to be able to play them. You'll have to normalize. What I'm saying is that you can use these residuals as temporary storage to avoid overflow and maintain dynamic range until you finally reach your output signal. This runs in realtime in embedded systems. It's cheaper than an FFT even. \$\endgroup\$ – Chris H Dec 3 '10 at 4:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.