I have always been uncertain as to which parameters are defined and which are being measured, or what variable comes before or after a calculation. Thanks to @Jiří Maier's answer above and @G36's link, but the concept only came clear after I drew up a sample circuit and plugged in some values:

The circuit is a diff-amp with CM error due solely to resistor tolerance as an example. There'll be an output offset caused by resistor mismatches, but you can't algebraically (symbolically) resolve differential voltage, CM voltage, or their corresponding gain \$ A_d \$ or \$ A_{cm} \$ as a function of resistor values or tolerances by itself.
Things make sense after we plug in numeric values. After which the equation can be manipulated into a composition of \$ V_{OUT(d)} \$ and \$ V_{OUT(\text{offset})} \$. While it is easy to see 98.04 as the differential gain, you can't say 0.04 is the CM gain because \$ V_2 \$ may not necessarily equate \$ V_{cm} \$ (or \$ {V_1 + V_2} \over 2 \$). The term \$ V_{OUT(\text{offset})} \$ as a whole indicates there is an offset to \$ V_{OUT(d)} \$, but \$ V_{OUT(\text{offset})} \neq A_{cm} V_{cm} \$.
0.04 can only be CM gain \$ A_{cm} \$ if and only if there is zero differential voltage (or \$ V_2 - V_1 = 0\$ ), which automatically implies \$ V_2 \$ equals to \$ V_{cm} \$. Only then you see the ratio of \$ {98.04 \over 0.04} = {{A_d \cdot V_{cm}} \over V_{OUT(cm)}} \$ and define it as CMRR. Only in this case for zero differential input can \$ V_{OUT(\text{offset})} = A_{cm} V_{cm} = V_{OUT(cm)} \$. To demonstrate:
If \$ V_2 = 3V \$ and \$ V_1 = 2.8V \$, the exact equation will yield
$$
\begin{align}
V_{OUT(total)} &= V_{OUT(d)} + V_{OUT{(\text{offset})}} \\
&= 98.04 \cdot (3-2.8) + 0.04 \cdot 2.8 = 19.72V \; ✓✓ \;
\end{align}
$$
My difficulty then was not realizing \$ V_{OUT} \neq A_d V_d + A_{cm} V_{cm} \$ !!! (I was misled by the terminologies used here into thinking they were equal), which otherwise would be off by 0.004V:
$$
\begin{align}
\;\;\;\;\;\;\;\;
V_{OUT(total)} &= V_{OUT(d)} + V_{OUT{(cm)}} \\
&= A_d V_d + A_{cm} V_{cm} \\
&= 98.04 \cdot (3-2.8) + 0.04 \cdot {{3+2.8} \over 2} = 19.724V \; ✘✘ \;
\end{align}
$$
To sum things up:
CMRR is defined and measured by \$ \text{CMRR} = {{A_d \cdot V_{cm}} \over V_{OUT(cm)}} \$ straightly for zero differential voltage (\$ V_d = 0 \$). And the relation \$ \text{CMRR} = {A_d \over A_{cm}} = {{A_d \cdot V_{cm}} \over V_{}} \$ again presumes zero differential voltage (\$ V_d = 0 \$).
When a signal composes of both differential and common mode voltage (\$ V_d \neq 0 \$), there is no straight forward way of calculating \$ V_{OUT} \$ in the form of an equation. And \$ V_{OUT} \neq A_d V_d + A_{cm} V_{cm} \$. Instead, by knowing your gain, you measure \$ V_{OUT(\text{offset})} \$ by adjusting the input voltages.
Often diff amp has unity gain and the gain is set at the buffer stage. \$ \; \text{CMRR} = {V_{cm} \over V_{OUT(cm)}} \$ only implies \$ A_d = 1 \$