Voltage is relative. One voltage level only makes sense if you have another voltage level to compare it to. If you would look at the signal wire alone you wouldn't get any wiser, you wouldn't measure a voltage. You need a reference to measure against, so that you can say that the signal is so-and-so-many volts, referenced to ground.
That referenced to ground is implied when we talk about voltage levels, unless otherwise indicated. Balanced signals are often symmetrical about ground, so that when on 1 wire the voltage goes to +1 V on the other wire it will go to -1 V. Nicely symmetrical. But the receiver won't look at one wire and see the +1 V; it will look at the voltage difference between the two wires, and then ground is not relevant. Such signals will most often be transmitted over twisted pair cables, where one wire is the positive signal and the other one the negative signal. There's no ground wire.
Kortuk wants to see a pretty picture. Again. (sigh)
Et voilà, une jolie image. Don't look at En-Ya, this is about my wall clock. How high does it hang? Most people will say "about 2 meters". That's because we usually will take the floor as reference level, our "ground". But my apartment is on the second floor, and if I take street level as reference the clock will be at 8 m above street level. A geographer may suggest 35 m above sea level. So "height" has no meaning without a reference, and with voltage it's the same thing.