Why does CMRR degrade in an instrumenation amplifier when the feedback resistor is increased, keeping all other resistances constant at a fixed value? What mathematical relationship links the feedback resistor and CMRR?
This problem has at its heart the differential amplifier: -
So, if V1 is grounded, the amplification of V2 is calculated by first recognizing that V2 is attenuated by R2 and Rg and, for the sake of argument, make R2 = Rg hence the attenuation is 2:1. The signal at the +input is then amplified by the factor 1 + Rf/Rg and, if Rg = Rf then the amplification is two and the overall amplification of V2 is unity.
On the V1 side, we know the amplification is -Rf/R1 and if R1 = Rf then the amplification is -1.
This means that if R1 = R2 = Rf = Rg and both V1 and V2 have exactly the same voltage applied, no matter what that input voltage is (within the constraints of the power rails), Vout remains at ground level (0 volts).
It can be further shown that this situation is also true if the ratio of Rf to R1 equals the ratio of Rg to R2 (do some math!).
Why CMRR degrades in Instrumentation Amplifier when R3 (only the feedback resistor) is increased keeping all other resistances constant at a fixed value?
For the reasons I mentioned above. It's not limited to InAmps but all differential amplifiers.
I assume you mean, as both instances of R3 are increased. (If you only change one of them, this is not a differential amplifier at all, so CMR will be very poor, as Andy aka explains.) In the idealized circuit, there is no reason why this should degrade CMR. However, in the real system it gets harder and harder to keep the two instances of R3 exactly matched as impedance levels increase. In particular, they will have capacitive strays that are difficult to control.