Assumption: By ceramic capacitors, you mean the low cost, no-name generic Y5V ones such as are cluttering up my parts boxes - not NP0/C0G capacitors with tight specifications
Generic ceramic capacitors (not NP0 / C0G) may exhibit some fairly strong non-linearity, with temperature, frequency and voltage. That last, the voltage related non-linearity, is easily handled by using ceramic capacitors rated for an order of magnitude or more greater than the voltage they will be used for, i.e. a 50 or 100 volt cercap for a 5 Volt circuit (Thanks to David Kessner for pointing this out).
Also, generic types of ceramic and mica capacitors act as tiny microphones, changing their capacitance and inducing voltage variations corresponding to incident sound.
This is a function of the piezoelectric effect of some dielectric materials: The accumulation of electric charge across such materials due to mechanical stress, in this case ambient sound. In effect, the capacitor behaves as a piezoelectric sensor, which is not its desired function on a feedback loop.
Both these factors, non-linearity and microphonic behavior, contribute to generic ceramics being not recommended for audio feedback paths, especially where aural quality is important. You should be fine with an NP0 / C0G ceramic capacitor with tighter tolerances. If you don't know what ceramic capacitor you have, it is safe to assume they aren't ideal.
While mathematically, a fractional percentage of distortion appears insignificant, the human ear can often distinguish very minuscule distortion in sound, especially on stringed instrument pure, extended notes.
Besides the scientific concerns above, there is also an entire school of thought among DIY audiophiles, who would swear by various types of capacitor, provenance of the power supply, the color of the PCB, or even the cloth weft and weave of a cable cover, as factors contributing to a non-quantifiable "feel" of a sound system.