You never want to conduct line frequency rectifier current through PE ground as this adds conductive noise. Using the transformer for high impedance and insulation from high voltage transients gives the audio better common mode noise rejection ratios (CMRR).
Grounding is used for noise reduction when it stays at the same voltage by not forcing noise current thru it but rather providing a low impedance reference to high impedance stray noise ingress thus attenuation occurs in shielded audio lines or low impedance speaker drivers.
There are occasions where RF can create audio noise in preamps so a capacitor can suppress this to the audio power 0V whether that is PE grounded or not.
Thus when using PE ground for computer audio, realize the PC power supply uses CM chokes and lots of filtering so that there is minimal audio band noise current on PE ground. Expect all peripherals to be floating and then rely on the host only for PE ground to suppress radiated noise thru the network or shielded cables.
When multiple source and destinations use PE grounds, you can get the "ground loop" noise due to the difference in ground voltage and current that caused that.
The term "ground" simply means 0V locally whereever you define that reference point.
Safety or Protective Earth (PE) ground is 0V in the earth and close enough elsewhere to be safe. So with interconnections you don't want to add noise current when audio is interconnected. The current should only be the intended audio.