According to this Application Note from Keysight, the difference between those two calibrations is as follows...
The TML port calibration (known as Single Mode in earlier versions
of ADS) is used when a transmission line will be connected to the
port. For this port calibration type, Momentum extends the
transmission line (or any edge of the layout in contact with the
port). A calibration standard (a transmission line of twice the
length) is simulated to de-embed the effect introduced by the
transmission line. This calibration technique reduces the impact of
higher-order modes around the source and adds the mutual coupling
effect of the transmission line current with the circuit. It is
recommended when the circuit is fed through a long transmission
There is also the option to use uncalibrated ports in Momentum. These
ports were previously called Internal ports in older versions of ADS,
but are now simply called None. Uncalibrated ports can be placed
anywhere within the circuit (not limited to boundaries between
dielectrics and conductors) and may excite a single point, edge or
area. Uncalibrated ports may be de ned using a single pin referenced
to implicit ground (Gnd), or by using multiple positive pins and/or
multiple negative pins. Sometimes, uncalibrated ports are the best
choice for modeling how devices are connected to a circuit.
So it seems that each calibration will set different boundary conditions in the interfaces (ports) of your coupler, and that's why you're getting different results.
Just select the calibration type that better suits how the coupler will actually be integrated in the rest of your circuit. If you plan to access the ports of your coupler through transmission lines, the results of the TML calibration may be more accurate and correlate better with actual measurements when you build it.