Further to Neil's answer:
For tiny amounts of over-power (e.g. when a machine is switched off) the excess power is consumed by the remaining connected loads. During over power, all passive loads exposed to a small amount of excessive voltage, thus drawing more current and dissipating more power. These tiny fluctuations are usually not regulated out. Rather, grid statistics provide for a generally constant power consumption over short terms.
With a sufficiently high number of loads (large neighbourhoods...) the fluctuation is a statistical effect with small variance, and thus does not cause damage. With a low number of loads (a portable generator), controlling and maintaining the output voltage for fluctuating loads is paramount.
As an aside, active loads (regulated loads, like LED, controlled motors...) generally respond differently and do not draw proportionately more current. Their power-draw rigidness does not ameliorate the over-voltage.
The duration of the over voltage is determined by the lag in controlling the primary power (up or down), and/or the lag in detecting this and shunting-in dummy loads.
The primary power lag is small for non-mechanical power supplies like solar cells. Switching off power is as fast as switching an array of power transistors or relays.
For mechanical supplies, like impeller-based generators (steam, diesel, hydro) the lag is a matter of mechanical momentum: how rapidly can you accelerate or decelerate the rotation of a heavy lump of metal.
A whole different question is maintaining the efficiency of power generation. Where does excess generated primary power go (heat, steam, water)?
Primary power delivery is controlled by the primary power consumption (e.g. the amount of fuel burn) times the efficiency or efficacy in transferring the mechanical power to electrical power.
The primary power consumption is relatively more difficult to regulate. For rapid momentary excess of power, primary power is "let off" through by-pass vents in the generator. This however results in an immediate loss of efficiency. For long term power reduction, generators are operated at lower power or they are "taken off the grid".
So back to the question, where does the excess power go?
- into existing loads (connected appliances, esp. passive loads)
- into dummy loads (shunted-in as needed)
- bled off (by-pass of mechanical power)
- regulated out (reduced primary power)