I get that each mV is equivalent to a bit in a 12-bit ADC, but why is 4.096V used so often? I've heard of this being from a "bridge". What does this mean and what is the historical significance?
It may or may not have historical significance, but if you have a perfect 12 bit DAC/ADC, each A/D count (or LSBit) corresponds to 1 mV, making some math easier.
FWIW, I personally see 2.5v being more common and I use internal bandgap references for non-critical measurements.
4.096 volts appears to have nothing in common with various Standard Cells, such as they Weston Cell. Even using Li, you'll not get much more than 3 volts from a standardizable chemical reaction.
The US. Legal (or NBS) Volt is presently deﬁned in terms of the atomic constants h (the Planck constant) and e (the elementary charge) via the ac Josepson effect. Critical to this is definition is the role of the Josephson junction which may be regarded as a frequency to voltage converter, where the frequency-to-voltage ratio is precisely equal to to the combination of physical constants 2e/h.... The U.S. legal definition of voltage is known to be smaller than the SI unit by about (9+-1) ppm...
There's no obvious conection to 4.096 volts there either, so I strongly suspect the value 4.096 is chosen for its convenience when using 12 bit ADC, rather than any underlying fundamental physical principle.
You run that 4.096 as the reference to your ADC, and as the power to your bridge. Then, if it varies with time or temp, the reading into your ADC stays the same. I've heard it called a radiometric measurement. The 4.096 gives you plenty of range on your 5V ADC, some headroom for your esd diodes and is commonly available.