Another consideration that leads to class D amplifiers not being used at high frequencies is the increasing proportion of the total time consumed by the (lossy) switching operation as frequency increases.
There is a limit to how quickly switching can be accomplished: if it can be done in 100ns, then 1/250 part of a 20kHz cycle is consumed by switching; one still has a reasonable number of graduations in the pulse width and thus the possible amplitude of an eventual 20kHz tone, so relatively low-order output filters can be used. At 200kHz, with switching alone taking 1/25th part of the cycle, amplitude resolution would be much reduced and higher-order filters would be required to limit the harmonic content of the output.
Perhaps switching could be accomplished in 10 rather than 100ns, but probably not much less than that, especially if dead time is provided so as to limit switching losses. At higher frequency, switching must (self-evidently) occur more often, and given that it takes a certain minimum time, then at some point the amplifier will need to switch often enough that it no longer has the benefit of being in the low-loss, purely on or off state for long enough to justify the class D topology. As frequency and power output increase, the efficiency benefit of class D over class AB becomes increasingly marginal.