I'll attempt an answer here but hopefully others can fill in some of the gaps in my own knowledge.

Grid following inverters are current sources using the grid as a voltage reference

The gist of it is that grid-following inverters act as current sources to maximize power output and rely on the inertia of the grid to maintain proper voltage and frequency. When the grid voltage and frequency deviate from accepted limits (per IEEE 1547), they disconnect from the grid.

If you were to try and "trick" the grid-following inverter with a simple ac power supply, it would not have the inertia necessary to provide a stable voltage and frequency, and the inverter would disconnect. Basically, the grid-following inverter would attempt to increase the current output to maximize power transfer, causing a rise in the system voltage which would exceed the capacity of your power supply's regulator. The rising voltage would trip the inverter offline (if it didn't fry the power supply first). The power supply would stabilize, the inverter would reconnect, and the process would start again.

In contrast, grid-forming inverters create their own reference

Grid forming inverters create their own reference voltage and current. They also attempt to maximize power transfer by increasing current output, but when the voltage rises near the limit they are designed to back off the current in order to maintain stable voltage and frequency.

Putting a grid-following inverter and a power supply back-to-back doesn't work because they aren't designed to operate together.

What is grid inertia?

The article "Inertia and the Power Grid: A Guide Without the Spin" (pdf) from the U.S. National Renewable Energy Lab gives a great explanation:

Inertia in power systems refers to the energy stored in large rotating generators and some industrial motors, which gives them the tendency to remain rotating. This stored energy can be particularly valuable when a large power plant fails, as it can temporarily make up for the power lost from the failed generator. This temporary response—which is typically available for a few seconds—allows the mechanical systems that control most power plants time to detect and respond to the failure.

The Wikipedia article on inertial response also gives a good overview, and goes on to discuss the decline in grid inertia as the penetration of inverter-based resources increases.

Some other references. From the U.S. National Renewable Energy Lab, "Grid-Forming Inverter Controls"

Most inverter controllers today are grid-following and built on the assumption that system voltage and frequency are regulated by inertial sources

From Du, Wei, Schneider, Kevin P., Tuffner, Francis K., Chen, Zhe, and Lasseter, Robert H.. 2019. "Modeling of Grid-Forming Inverters for Transient Stability Simulations of an all Inverter-based Distribution System"

A. Grid-Following Concept. Currently, most grid-connected, inverter-based DERs use grid-following control, which typically uses a phase-lock-loop (PLL) and a current control loop to achieve fast control of the inverter’s output currents. Grid-following control makes the voltage source inverter behave approximately like a current source, as shown in Fig. 4 (a). The advantage of this control is that the currents can be quickly regulated. However, because grid-following control does not control the voltage and frequency, it relies on an external voltage source to provide the voltage and frequency references. During load disturbances, grid-following inverters maintain their output power approximately constant.

B. Grid-Forming Concept. In contrast, grid-forming control controls the voltage and frequency of the inverter, making the voltage source inverter behave approximately like a voltage source, as shown in Fig. 4 (b). Because the voltage and frequency remain constant, the grid-forming inverters can work in stand-alone modes and track the loads. To achieve parallel operation of multiple grid-forming inverters, different control strategies have been proposed, including droop control, virtual oscillator control, and virtual synchronous machines, etc.