you can implement something called a Crow Bar circuit, you can learn all the wonderful things about these by reading the Art of Electronics by Horowitz and Hill (1989) and they are cheap and simple to set up. The crow bar circuit clamps the node it's attached to when the voltage rises above the defined clamping voltage - and continues to conduct until the fault condition is removed. This is often used as over-voltage connections from out-side sources, and is much safer when there is a fuse or sacrificial element between the high/over-voltage source and the crow bar circuit.
In your case, it will at least pull the voltage down to the clamping voltage, but the circuit itself is not immune to damage and your crow bar (if the input source is powerful enough) may fail either to a complete short circuit, or in open circuit where your "protection" is now void.
edit: here's a wikipedia article