The ground connection is regarded as a big sponge for interference i.e. it soaks up that interference with barely any change in potential along its length. Therefore the single wire can be regarded as the most susceptible part of the transmission link.
Taking it one step further - interference hitting a long thin wire has a much more significant effect on the wire's end-to-end voltage than interference hitting a vast area of "fat" ground so, even if ground was (slightly) affected, the biggest point of vulnerability (by far) is the long thin wire.
And, because the long thin wire doesn't deal with noise as well as ground, you get a differential noise produced that cannot be cancelled by a differential receiver amplifier.
However, if the grounded signal source were connected to the far-away differential receiver by two wires (one being grounded at the source) AND the driving impedance of the source was very low (a few ohms max), then the two wires (of which one still represents ground) will likely receive the same interference signal and now, a differential amplifier will work. But, this only works if the wire that is remotely grounded at the source is handled electrically in the same way as the signal wire at the receiver. In other words, if you went and grounded that wire again you'd make a mess of things.
Here is a more general approach: -

Because the receiver is input impedance balanced AND the source is driving the signal line with the same impedance as seen in the grounded line then we have a balanced line driver circuit.