For such measurements done with a sampling oscilloscope, the time base of the scope and of the function generator must be synchronized to a common reference.
In a lab where you deal with precise frequency measurements, you'll need a common 10 MHz reference, with at least two outputs, or a distribution amplifier, and this output is then fed to all the instruments that have a sampling clock of any sort, as well to frequency counters etc.
Typically, such a lab reference is a rubidium or crystal oscillator disciplined by GPS, called a GPSDO for GPS-disciplined oscillator. You don't need that - a simple 10 MHz quartz oscillator will do just fine, with two line drivers on the output. Test equipment typically terminates the 10 MHz reference inputs in 50 ohms, thus the need for the line drivers/output amps. Typically, a lab frequency reference either comes with a distribution amp with multiple outputs, or it feeds its sole output to one.
Scopes costing below $10k probably won't have a 10 MHz frequency reference input. The DS1104Z doesn't, unfortunately, and it doesn't have an AWG either.
But many scopes in that price range, e.g SDS2104X, have an AWG that can generate a 10 MHz sine or square output. This would then be fed to the function generator's reference frequency input. The AWG is usually running from the oscillator that feeds the sampling system, so that works. It certainly works for the SDS2104X and SDG2042X combo.
I haven't looked into clock sources inside of a DS1104Z, but if the SoC and the FPGA run from the same oscillator, then there is a "third option" that could perhaps work:
The USB host's 125us-period "microframe" SOF signal can be detected and converted to a 4kHz square wave. A 10MHz square wave generator could be phase-locked to this 4kHz reference, and used to feed the AWG's clock reference input. This will have more jitter, so the demodulated waveform you'll observe will be more noisy than what you have seen so far, but it'd work.
Detecting the SOF and outputting it as a square wave can be done with a small FPGA, or a microcontroller with a USB device peripheral.
You're massively undersampling the signal - both scope screenshots you show are sampled at 100 kHz. That's fine: you're using Nyquist sampling - a valid technique to demodulate low-frequency drift of frequency sources. As long as valid metrological techniques are used!
What you're observing is the beating of the oscilloscope's reference oscillator with the generator's reference oscillator. Or, more directly: the signal you observe is the relative phase between the generator's sampling clock and the oscilloscope's sampling clock.
You will see such waveforms no matter what kind of a generator you're using - as long as the generator and the scope's time bases are not mutually phase-locked, or locked to a common reference.
You'd see it using a purely analog function generator - it would likely be much worse, since simple analog function generators aren't super stable in frequency - certainly much less stable than a crystal oscillator.
If you used a (RF) signal generator that phase-locks the output frequency to a reference, and fed the reference to the scope for use as the sampling clock reference, then you'd see the flat line you expect. But you're not using a signal generator, but a function/waveform generator, a digital one - since its output is inherently sampled, there is a sampling clock involved, and as long as you get that clock in sync with the scope's clock, you're good.
Now, if your scope does not have a 10 MHz clock reference input, you're somewhat out of luck, unless you'd feel comfortable opening the scope, disconnecting the oscillator used for the sampler, and feeding the same frequency from an external generator phase-locked to a 10 MHz reference shared with the ARB/AWG you're using. Any competent AWG will have a 10 MHz external frequency reference input. You could also have a little adapter board with a frequency divider that would scale down the sampling clock oscillator's output down to 10MHz, and this would be fed to the AWG's frequency reference input.
If the scope and the AWG offer no clock reference inputs, then not all is lost. If this was done in a professional setting, you'd just get a better scope and AWG, since without clock reference such tools are not very versatile, as you've found out.
You'd have to modify the AWG and the scope to buffer and bring out their sampling clocks to an external connector. You'd then use an external phase detector with analog output to generate a phase difference signal. That signal would be passed back to the AWG's modulation input, with Phase Modulation selected. The phase signal would require appropriate scaling/offsetting to match the imperfect inputs on the ARB, and you'd get artifacts each time the phase wrapped around.
This is the "modulation hack" you were originally proposing. It's not practical.
