Assuming the op amp is ideal, the virtual ground will be maintained. The ideal op amp has infinite gain and is infinitely fast.
However, any real op amp will not have infinite gain. It will not be infinitely fast. And more than likely it will have some limited output slew rate. EDIT: Another important thing to consider is that a practical op amp has limited output voltage range. You will likely see the output signal clipping terribly for high-frequencies.
So, using a real op amp, the op amp will not be able to keep its negative terminal at 0 V at high frequencies. You can see this by connecting a square wave at the input of the differentiator. Just after the rising and falling edges, you will likely see the op amp's negative terminal jump in the direction of the edge of the input square wave. I.e. if the square wave is transitioning from low to high, you will see a positive spike at the inverting terminal of your real op amp, and a negative spike on the high to low transition.