In EEVBlog #1116, Dave discusses a method to remove power supply ripple, and goes on to show (see 5:17 to 6:15) that you cannot count on linear regulators to remove your input ripple. He gave a concrete example in the lab: at 10 kHz input ripple and MCP1700 (a CMOS LDO), as demonstrated on the 'scope, the ripple largely passes through.
While the rest of the video is meticulously explained, I feel that he present this example in a bit of a cherry-picked manner and omitted relevant details. I remember doing exactly the thing he warns against: I had a class-A headphone amplifier, which, when powered via a specific el-cheapo wall-wart at 12V, had a whistling sound on the output, caused by the switching noise of the power supply. In that occasion I lowered and cleaned the input voltage with a LM317, which completely removed the noise.
Note I'm not saying Dave is wrong - his warning is that a linear regulator, and a LDO in particular, may not solve your problems.
I have enough intuition to guess that what he talks about likely applies mostly to LDOs, since I've heard they can have stability issues and I guess the internal compensation against oscillation makes their pass element somewhat inert, so at frequencies like the 10 kHz he tests with, things can be quite bad. I don't see how they would fail the same test at 50-120 Hz, since this is a very common usage scenario which the IC designers likely thought about.
Do all linear regulators perform poorly — say, have ripple rejection less than 15dB — at some combination of frequency and load current? Assuming other conditions aren't super-bad, i.e. not talking about 125°C and/or input voltage touching the dropout zone? On a related note, is there a linear IC design, which is particularly good at rejecting input ripple all the way up to 500 kHz?