Silver has a lower resistivity than gold and is cheaper, so why are high end audio components gold plated?
Gold is highly resistant to corrosion or oxidation, so prevents poor connections from those sources.It is also fairly soft, so the mating surfaces deform slightly, increasing contact area to reduce resistance. The gold plating is very thin, so the added resistance from the gold is easily overcome by its other properties.
Note that gold is only needed on the actual contact areas - gold plating (or colour) on the body of the connector is only there to attract the gullible.
Many commercial (not audiophile) connectors have selective gold plating on the contacts - the gold is only placed where it really matters.
Tarnished contacts cause distortion. You can really hear the audio quality degradation; it is not some audio-fool nonsense, like directional speaker cables or two hundred dollar amplifier power cords.
Even a dyed-in-the-wool objective measurement man like like Doug Self acknowledges this problem, in no uncertain terms, right in the middle of his treatise on "Subjectivism":
Corrosion is often blamed for subtle signal degradation at switch and connector contacts. By far the most common form of contact degradation is the formation of an insulating sulphide layer on silver contacts, derived from hydrogen sulphide air pollution; the problem seems to have become worse in recent years. This typically cuts the signal altogether, except when signal peaks temporarily punch through the sulphide layer. The effect is gross and completely inapplicable to theories of subtle degradation. Gold-plating is the only certain cure. It costs money. A switch with gold-flashed contacts can cost five times as much as the silver version. [Source] [Bold emphasis mine].
Other metals like silver will do, but the contacts will require regular polishing to perform their best. Gold is more maintenance-free.
I own a bunch of old audio and guitar gear, and I can tell you that if you put a little bit of toothpaste on a cotton swab and vigorously polish the inside of a 1/4" phone jack on some old amplifier or signal processor, it comes out black. The sound quality improves drastically.
An important thing to remember is that when dissimilar metals are joined, and there is a thermal flux, a voltage is generated due to the thermoelectric effect. This voltage is called a thermal EMF.
A good discussion of this effect can be found in the classic Linear Technology application note AN-86 by the late Jim Williams. See in particular page 48 for a table of thermoelectric EMFs.
From that table, note that copper-copper junctions have the lowest thermal EMF of less than 0.2 uV/°C BUT notice that copper-copper oxide junctions have the highest thermal EMF of over 1000 uV/°C. Copper-gold junctions have a reasonable thermal EMF of 0.3uV/°C.
So, in precision voltage / current / resistance measurements we often use cleaned copper-copper junctions or copper-gold junctions to minimize thermal EMFs.
That said, in audio applications, if the contributed thermal EMF was 100uV, if the full scale signal is 15 volts, that's only 6.6ppm (parts per million). That's nearly -120dB. It's pretty hard to make the case that a human can hear 6ppm / -120dB.
Some professional audio gear does use silver or silver alloy contacts (or did : the items I am thinking about are well and truly obsolete!) - such equipment served the broadcast audio for decades - BUT it was regularly and thoroughly maintained, with metal polish being an essential item of the toolkit.
The rotary "stud faders" in BBC mixer desks up until the 1970s were of this type : not variable resistors but switched attenuators with something like 1dB steps, a huge array of precision resistors, and solid silver contacts. These lasted until the stereo era.
Prototype stereo "stud faders" were tried - but however accurately they were made, the two channels switched gain at slightly different times, and the "blurring" in the stereo image during fades was judged to be quite unpleasant.
Here is a description of a mono one - as you can see, rapid maintenance was an important design consideration. (I cannot embed the picture, it appears to be protected) The contacts look pinkish; they need cleaning to show their true colour.
So silver has a long history in audio, and not all of it can be dismissed as woolly headed nonsense (though there is plenty of that too!).
But gold is easier to deal with and, while lower conductivity, quite good enough for most purposes and overall better for domestic equipment. Unless you are so dedicated to perfect sound that you are willing to overhaul every connector and switch contact on something like a regular six-monthly schedule.