simulate this circuit – Schematic created using CircuitLab
This circuit assumes that the net DC voltage on both input and output is zero. If there is any offset, you will need to add coupling capacitors to block the DC.
The probable reason you are getting clicks with your circuit is that the switching action is too fast. I'm going to guess that the clicks are almost inaudible if there is no audio signal but become more noticeable as the audio level increases.
What you want is an attenuator that ramps from full to off as well as ramping from off back to full on.
You can play with the choice of MOSFET. The 2N7000 is an old part that doesn't have very good Rds-on ratings - this will limit the amount of attenuation that you get. But it has a low threshold voltage - this is important if you are controlling the MOSFET from an Arduino.
There are many suitable MOSFETs available that will work better.
Play with the values of R3 & C1 to change the ramp time.