An open-loop op-amp has very high gain and, because most op-amps have to be stable when operated at unity gain (with full negative feedback) it has "compensation" built into the amplifier. For the LM324 it's the capacitor in the red box: -
Cc isn't very big (a few pF) but it makes the open-loop response appear like that obtained from an ideal op-amp integrator. Cc is the dominant component that affects the gain diminishing from the DC gain (several hundred thousand to a million or more) to unity (at several hundred thousand kHz to a MHz or more). A decent picture of another op-amp's open loop gain and phase response is this from the TL081 data sheet: -
Pretty much above a frequency of 10 Hz it's got the characteristic 6 dB/octave (or 20 dB/decade) frequency roll-off and by the time the frequency reaches a little over 1 MHz, gain has "linearly" fallen to unity.
This is exactly what you would get from an integrator. So let's say you had a perfect op-amp and decided to make an "imperfect" op-amp (or integrator). You would have an input resistor and a feedback capacitor like this: -
Also shown is a feedback resistor (in parallel with C) so that it "models" the magnitude of the DC open loop gain (R2/R1). So, from DC up to a few Hz, the dominant components are R1 and R2 but, gradually C starts to have more influence and this moves the phase shift from zero to what we would expect from an integrator (90 degrees).
So, imagine that R2 was much, much lower (like for a typical feedback resistor in a closed-loop amplifier) and ask yourself at what frequency the phase shift starts to head towards 90 degrees. It would be much, much higher. In other words if R2 is dominating the impedance of C (or Cc) then the phase shift is as you would expect from an inverting op-amp configuration.