I might not have drawn the second one right, but imagine if it was a live wire in a bathtub going through the water and the plumbing to ground instead of returning to neutral, except way more volts.
Your first drawing is an isolated DC power supply such as a battery with no connection to ground. The voltage measured between the terminals will be the battery voltage.
Your second supply has two breaks in the circuit. The supply is still isolated so no current will flow. The voltage measured between the terminals will be zero as the battery negative is not connected to anything. You could test this with a multimeter and a battery: connect one meter lead to an earthed point (cooker / fridge chassis) and the other lead to the positive of a battery (with no connection on the negative).
Then how does my bathtub analogy work? Clearly there is current running there. So does that mean that that current comes back into the source through the ground?
Figure 1. Your domestic supply is earthed at the supply transformer. Earthing the bath ensures that in a fault the voltage of the bath is kept close to earth potential.
With a high-current earth fault the fuse will blow. RCD / GFCI devices add an additional level of protection.
Also doesn't real current run from the negative,
No. The convention is that current flows from positive to negative. It's true that in metals that the charge carriers are electrons and that they drift in the opposite direction to conventional current. In other circumstances current flows in positive ions. Don't get hung up on this.
... so if I connect the positive of the battery to the fridge, there would be 2 reasons for why the multimeter reads 0.
No, we're reading voltages so it doesn't matter whether current is flowing or not.