I have designed a 2nd order Butterworth filter with MATLAB fdatool for VHDL implementation. the cuttoff frequency is less than Fs/10, which should be okay since fdatool uses prewarping. Indeed, the floating point implementation works fine. However, after quantization to fixed point, even with very large word lengths (64 bit), the filter does not work as expected. Is there a way of fixing this? or is there another way of approximating a butterwoth response which will work with fixed point ?
The question is how did you do the shift to fixed point. The main key here is that you're trying to preserve all the dynamic range.
If all your filter constants are between 0 and 5, let's say, if you use 64bit you will have very little dynamic range because it is impossible to do 1.3, you can just do integers.
What is typically done, for example, is to use 16 bit as Q1.15. This means that you have a range from -1 to 0.999969482421875. Here each bit corresponde to 2^(-15).
A consequence of this is that you need to scale your input so that your maximum is constrained to -1/1. If you're reading values from an ADC, for example, that's quite straightforward to do because you know the maximum value that can be read.
For a second order filter 16bit should be more than enough dynamic range, unless you're frequency range is also huge.
I tried using int16 an in32 types for the Signal sample, and I found that when the wordlength has enough overhead (sign bits between the Sign bit and the MSB) the filter works fine, so apparently it is an overflow problem. What I did was scale the signal up: the float signal is between +-1, so for int16 I multiplied it by 2^5, and for int32 by 2^21, and tested the filter- it worked fine. in some cases less overhead was necessary,for the 1500 Hz filter, I needed 4 extra bits, and for the 250 Hz I needed 10 extra bits.