Imagine we have a time-domain signal which is a voltage signal. Now if we sample this at the correct rate, and taking the Fourier transform gives us the magnitudes of various frequency components present in the signal (within the context of sampling rate).
More or less. The Fourrier Transform gives us the amplitude (which is complex) of the frequency components.
Taking the modulus squared of this quantity will give us power/Hz, correct?
More lor less. It gives you power per bin. You have to correct for the bandwidth of the bins to get power/Hz. But you do lose the phase and get to a magnitude.
On the other hand, taking the Fourier transform of the autocorrelation function (rather than the direct transform as discussed earlier) gives the PSD in power/Hz as well.
Yes, given the same caveat about power/Hz.
So what is the difference between these two? Why/when should I do one over the other?
They are slightly different, because they use different portions of the input signal. Let's say we want a 100Hz resolution spectrum, so we transform 10mS of time data. In the direct method, we use 10mS. In the autocorrelation (AC) method, we use 20mS, autocorrelate leaving us only the central 10mS as valid data, then transform that. With a stationary signal, the results are very very similar.
Which method a digital scope use for its FFT function?
Modern scopes tend to use the direct form, though there is much more to taking the spectrum than you think.
The first operation is to select a finite portion from the time record. The DFT (Discrete Fourrier Transform, for which the FFT or Fast Fourrier Transform is an efficient algorithm producing identical results) interprets this as cyclic, so this is where all the spectral leakage damage gets be done.
The next operation is to repair the spectral leakage by using a window function, a Hamming window is a popular one. Different windows have different tradeoffs in terms of suppression of spectral leakage, widening of the bin bandwidth, and amplitude flatness across the width of a bin, but they all look fairly Gaussian to the untrained eye. Choice of an appropriate window is often the secret sauce that gives you a usable or useless spectrum.
Now take the FFT. This is the standard bit. Square to power. Normalise for bandwidth, and for any input reference levels and gain.
If you're transforming a line signal, you're done. If you're transforming noise, then you need to rinse and repeat, with further time records, and average the spectrums, to reduce the variance of the noise.
Often at this point, you'll get the option to run a marker across the spectrum, to read out individual frequencies and magnitudes. The simplest (zero effort) marker will convert the bin centre frequency to Hz, which is why you'll sometimes get binary frequencies like 23.375Hz when you're expecting some other resolution. The amplitude error for frequencies off the bin centre can often be 1dB for typical windows, to 3dB for a rectangular window (rectangular does have its uses, but only if you know exactly what you're doing). A better marker will estimate signal power using the zeroth moment, and signal frequency using the first moment, of the powers of any individual resolved peak across the width of its broadened line, often giving usable resolution to 0.001dB and 1/1000th of a bin width. If your instrument doesn't give you this, you can post process a raw dump of the spectrum with a spreadsheet or program to get it.
The marker function will usually have a selectable normalisation to either total power or per Hz power, and with any luck, referenced to volts or dBm at the front panel.
If you did want to do the spectrum via an AC, you wouldn't do it directly, multiplying the terms in the input, as the effort involved is O(n^2). You'd use a Fourrier Convolution using O(nlog(n)) effort to compute it. So if you're transforming to do the AC, you might as well just transform to get the spectrum. The AC method was quite popular before the FFT algorithm had been discovered, especially if only a few spectral points were required.