My biggest challenge with this question is its ridiculously steep roll-off rate.
I am assuming that the signal is undetectable if its gain is -20 dB. This means that, within the transition band of 200 Hz, the signal strength needs to drop by 20 dB.
If my calculations are correct, this filter requires a roll-off rate of 1200 dB/dec. That requires 60 poles, which is obviously not feasible.
I would like to use an analog active filter with minimal ripple in the pass band. A large phase shift is not too important.
One potential solution is to use a notch filter at 5.2 kHz. However, frequencies above the bandwidth of the notch filter are still not sufficiently filtered.
Please point out any flaws in my logic and or propose potential solutions. Thank you.