Your amplifier will have an "input common mode range" specification. While it measures the voltage difference between its inputs, both inputs must be inside the amp's common mode range (relative to its ground) for it to work.
Linking the grounds ensures this. It also allows use of a shielded cable.
Usually, but not always, this range is inside the amp's power supplies, minus a few volts.
If the grounds are not linked, then very high voltages can exist, due to static charges.
Sometimes you cannot link the grounds because you want isolation. In this case, your instrumentation amp has to be a transformer... or you can use an amplifier on the sensor side and optical isolation, for instance.
The resistors are there to allow the amp's input bias currents to go somewhere, and also to ensure the input common mode range is respected.
Now what is the voltage V_AG and V_BG here?
There is no way to know. In reality, it would depend on leakage currents, static charges, input bias currents, input impedance of the amp... If the amp is a standard instrumentation amp, it would not work, since the input bias currents would have nowhere to go, and this would set the input common to a random value, probably outside the admissible range.
Responding to your comment:
There are several ways to link the grounds:
- Wire and Cable shields make an explicit connection
- Earth can make an implicit connection
If all boxes are floating (or only one is earthed) then the usual way is to use cable shields or wires. Simple and effective.
"Earth" is only available if both ends if the connection are earthed, of course. Since the potential of "Earth" may vary, if the cable is long (ie, between buildings) then large currents can flow in its "ground" connection. In this case, it is better to have only one equipment earthed. This is why Ethernet uses transformers, for example.
When both equipments are earthed, you also get a free ground loop. In this case, you want any current flowing into this ground loop to flow into the proper parts of your circuit (for example, the thick aluminium plate at the back which carries all connectors) and not in a track on your PCB. This is the case of "pin 1 problems" for audio gear, for example.
When both equipments are earthed, it is also possible not to connect their grounds together at DC using cable shields. A capacitive connection is still mandatory, of course, or else the shield becomes useless at HF.
Input bias currents flow from both amp inputs. Their direction depends on the amp. If they aren't allowed to return to amp's ground, this means the inputs are floating. In this case, of course no bias current will flow since it has nowhere to go, but then the amp won't work.
For example, if the input stage is PNP, then (ignoring schemes like bias current compensation) the base current flows out of the inputs. Without a return path, there will be no base current, the input transistors with turn off, and the amp won't work, its output will most likely clip.
So, a more correct way to write "Input bias currents flow from both amp inputs" would be: "If input bias currents are not allowed to flow and return to the amp's ground, then the amp will not work."
Now, if you use a FET input amp, bias current is zero (no base current), but there is still a tiny amount of leakage current coming out (or into) the inputs. If it's a JFET, its gate will also have leakage.
Now, if grounds are not connected, there is still a very high, but not infinite resistance (like air, or a bit of FR-4)... then you will have :
Input voltage = (tiny leakage current) * (enormous insulation resistance between unconnected grounds)
Whichever wins, wins, but since this depends on temperature and humidity, your inputs will clip against one rail or the other, and the amp won't work.