Modern cell phone chargers operate a very high frequency- well above audible even for small dogs.
However, what I suspect is happening is that your inductor is perhaps 1mH and your adapter is perhaps capable of 600mA before overcurrent protection kicks in.
So, when you connect a fixed voltage to an inductor in series with a resistor, the current increases with time constant L/R towards Vin/R = 0.65A (allowing a bit of resistance for the inductor- and the DCR of the speaker won't be exactly 8 ohms).
This time constant (based on my guess as to the inductance) is of the order of 125usec, so if the overcurrent protection kicks out then restarts, the frequency of restarting should be in the several kHz range.
You don't hear the sound when the speaker is connected directly across the power supply because the restart frequency is above the audible range.
To confirm this, look at the voltage and current waveforms with an oscilloscope.