The relays are the best in terms of signal quality. If you're going for state-of-the art performance, you may have no choice. You'll find them in many precision measuring instruments, including ones that I've designed, and including many (if not most) high-end bench instruments such as multimeters and function generators. There's just no comparison if you need the performance.
In between the two (relays and cheap analog switches) are more expensive high-quality analog switches that are more completely specified, such as ADGxxx. They are better protected, have much lower on-resistance and better specified leakage (1nA or better in many cases).
Analog switches suffer from relatively large maximum leakage (to power supply rails), more cross-talk between channels (unspecified maximum), feedthrough (off switch still transmits some signal ), relatively large series resistance that varies with voltage and temperature (and unit-to-unit). The variability with voltage means that there will also be distortion created that depends on the load resistance. The variability with temperature means that offset and gain will vary because of leakage current and load resistance, so even low frequency applications don't get off scott free.
So, look at the numbers. There are many, many circuits where the analog switch is good enough or more than good enough, and a few where you really cannot live with the many deficiencies.
Note that in typical (lying) datasheet fashion, they use a relatively high load resistance to specify the distortion and a relatively low load resistance to specify the crosstalk and feedthrough. If you increase the load resistance to 10K, the feedthrough could be as much as 20dB worse (10x worse), so at only a few MHz, you could see a sizable fraction of the input on adjacent channels. One way to avoid that is to use two switches in series and shunt the mid point to ground. Also works for relays!
If you don't mind the lack of protection against overvoltage, and the poor or missing "worst case" specifications (meaning no guarantees if you get 1uA leakage instead of the 100nA you're expecting), the 405x series of switches is really not all that bad with careful design, and you can get pretty good performance out of them.