OK, lets simplify this; If the circuit was in a metal box, say a transistor radio, on a plastic table, (no scope connected).
If you took a voltmeter, measured voltage from any part of the radio's circuit, to let's say - a water pipe (or anything in the room but not on the radio), you would read nothing. The battery, and circuit, does not 'see' the outside world, or earth ground. NOW, if you connect the scope ground to ANY part of the radio, lets just say--- the positive battery post. NOW you have introduced a connection to ground (of the outside world, and your AC powered test equipment most importantly). The outside world is tied at that one point now.
Everything in that radio will now have a voltage with respect to earth ground, (or anything electrically powered from AC or grounded in some way).
That point is now 'earth ground voltage'. Everything else connected in that circuitry will now have some sort of reading. The readings to each other will still be the same within the circuit, but you will also be able to get readings of how much it is 'above' or 'below' the point you have now grounded; in this case, everything will read 'below' earth ground (negative).
In this example, with scope grounded to the positive, no current flows if it is the ONLY ground point connected. The negative battery power cannot get to the positive, except through the radio's circuit as normal.
ONLY if you ground the radio chassis in some way; say you accidentally put a second scope probe ground on the battery NEGATIVE, or if you put the box on a metal grounded table (or touched the radio chassis and the scope case at the same time); THEN the battery power has a 'ground' route back to the positive; you have a short (or ground loop).
[think 'antenna' or cable TV feed that is grounded!]
So in this example you connected your scope probe to battery '+', then the battery NEGATIVE will NOT read '0'volts, but will read minus x volts (the batteries voltage). And everything in between will read somewhere between 0V and negative battery's voltage.
(Kind of upside down, so to speak).
If you had connected the scope ground to the battery negative, then everything would read 'right side up' as you are accustom to, and also to readings taken with no grounds attached as with a voltmeter and radio alone.
If you grounded your scope 'in between' somewhere in the circuitry; then your scopes readings would be some 'above' and some 'below' ground level (DC-wise). Remember, AC waveforms are all going to be the same. Technically this would be referred to as DC-offset considerations.
You won't have any shorts (or ground-loops) as long as you ONLY have ONE defining ground connection (and chassis is NOT grounded, that would be TWO ground sources in this example).
As long as there is only one connection point* (between the two 'worlds'), or no connection, there will be no short circuit.
*Obviously, you can have more than one ground, as long as they ALL connect to the same common place electrically speaking.
Note; the metal box could be connected to battery negative (typical in traditional circuitry for shielding and current return), battery positive (I repair vintage Germanium transistor circuits where all the voltages are reversed, uses negative 'hot', and positive 'ground'), or box could be connected to nothing.
[think of a vintage Dodge with positive ground battery; it didn't short out because it's bumper touched a parking pole or if parked next to a negative grounded car! Only if you tied an external ground into their circuits, or tied their circuits together, would you get fireworks. PS, You CAN connect the Dodge's negative 'HOT' to the negative of a 'negative grounded' car, and it's positive 'ground' to positive; just don't let the bumpers touch!]