You show a 15 V drive voltage to the MOSFET gates. Your text says this comes from a push switch. It would be valuable to edit the schematic to show this switch to 15 V.
What you don't have (as shown) but need is a gate pull-down resistor. This is to discharge the gate-source capacitance (usually Ciss in datasheet) quickly, to disable the MOSFETs when the switch is released. That stops the MOSFETs being left partly on and drawing enough current to dissipate power to damage themselves, as well as not turn off as intended.
The resistor value should be low enough to discharge the two Ciss quickly, high enough not to dissipate excessive power when the switch is pressed.
Try (a) getting rid of the Zeners and 10K then (b) adding a 10K pull-down to your 15V node.
Even if this is not the root cause of your problem, it is a necessary and valuable circuit correction to make.
From its datasheet, the LTL-307EE LED has a forward voltage drop of 2..2.6 V at 20 mA. It therefore needs a current limit resistor. Otherwise it will draw excessive current that could destroy the MOSFET, though I'd expect the LED to go first but nonetheless...
It would also benefit from a much lower supply rail, to reduce the power dissipation in that resistor.
Add a 1K2 1 W resistor in series with each LED.