As noted by others, the circuit you've implemented only provides reverse-polarity protection. I've seen this layout called an "ideal diode" MOSFET because the voltage drop is in the tens of mV, ten times lower than even a signal Schottky could provide. (Power diodes that could handle your load current, of course, are even more lossy.)
It works because of the body diode in the pMOS. Under proper polarity, the body diode conducts. The zener diode / resistor bridge then pulls down the gate voltage relative to the source, which biases the pMOS well into its operating region, where many power pMOS devices will have an on resistance of a couple dozen mΩ. Under reverse polarity, the body diode blocks, the pMOS gate voltage remains equal to the source, and no current can flow.
The solution to the overvoltage problem requires a second MOSFET. Because the pMOS in your circuit is "backwards", the body diode will always conduct if the drain-source voltage exceeds a minimal threshold. A second power MOSFET, connected "normally" and itself controlled by some zener diode reference voltage, can switch off the circuit when necessary.

simulate this circuit – Schematic created using CircuitLab
In this circuit, M1/D1/R1 form the reverse voltage protection circuit you already have. M2 is the load switching MOSFET which, under normal circumstances, is biased deep in its conducting region by the D3/R3 divider, with a source-gate voltage of 10 V.
M3 is a low-power MOSFET. Under normal circumstances, D2 is not conducting, R2 pulls M3's gate voltage up to its source voltage, and M3 is off. However, when the D2 cathode voltage exceeds its zener voltage (in this case chosen to be a little higher than 24 V), D2 starts to conduct, clamping the M3 gate voltage. Once M3's source-gate voltage drop is high enough, M3 conducts, D3 is bypassed, M2's source-gate voltage drops to zero, and the circuit shuts off.
Resistor values and zener voltages are for illustration. You'll need to match your resistors, zeners, and MOSFETs carefully to make sure your circuit doesn't turn off prematurely. A slight overvoltage combined with poorly-chosen components may result in a not-completely-off M2, which will significantly increase its on resistance, and thus its temperature, in a way you may not have expected.