The leftmost oscillator coil induces hopefully the same signal to the mid and rightmost (=R) coils. The mid and R coils are in series so that the caught sum signal is zero if there's no metal. If someone brings a piece of metal near the coils, it very unlikely affects (via eddy currents) identically to what's caught by mid and R coils. The balance vanishes and the detector line starts to get signal from the mid and R coil combination.
You can simulate the detector (a synchronous one) system simply by feeding the 5500 Hz oscillator signal to the +input of IC2-A. Do not cut its dc-path, R5 is a must.
To simulate the coil system you must know the coil inductances and their mutual inductances - total 6 different inductance values. I'm afraid that a direct measurement needs advanced instruments and calculating it needs advanced math knowledge, both beyond what I have. But if you know those inductances you simply change for ex the mutual inductance between the leftmost and mid coil.
You can measure from a built practical circuit how much mid and R coils catch.
How much a given piece of metal can cause mutual inductance imbalance at the operating frequency - it can be measured. It can be also calculated numerically with full 3D electromagnetic field simulation system. Again something quite advanced.
Not asked: There's many other metal detection systems. Some of them need only one coil.
