I replaced R2 with a P-channel MOSFET Q3, which discharges the capacitor C1 quickly, improving the load switch cut-off time.
I assume you have a 3.3V or 5V digital enable signal, so I changed Q1's role into that of a driver for Q3's gate, to retain compatibility with such a signal. It has the unfortunate side effect of inverting the active level, so I've called it "INH", for inhibit, instead.

simulate this circuit – Schematic created using CircuitLab
Here are plots of the inhibit signal and the resulting output current:

Addendum #1
@tobalt made the aposite observation that neither the original circuit nor mine above mitigates inrush current in the case where Vin is applied while E/INH is intially is set to enable output. That's true, as shown here, where I hold INH low and step Vin:

He suggested a quick fix, with a capacitor across Q1. Here is my circuit modified in that way, with the resulting output current plot, working as @tobalt predicted:

simulate this circuit

Addendum #2
As @tobalt also points out, if Q2's threshold Vgs is lower than that of Q3, Q2 can turn on when Vin lies between those two voltages, regardless of the state of the INH input and Q1. The situation can be modelled and demonstrated by removing Q1 and other unrelated parts, and holding Q3's gate at 0V (as if INH was asserted):

simulate this circuit
Both gates are effectively at 0V until Q3 begins to conduct, and therefore one will switch on prior to the other. If the first to switch on is Q3, then there's no problem, Q2 will be inhibited throughout the rise of Vin. Otherwise there's a window of Vin during which Q2 can conduct even though INH is asserted.
Here I have chosen transistors where Q2's threshold voltage is lower than Q3's, and as the resulting plots show, Q2 begins to conduct long before Q3, before Q3 is able to pinch off its gate-source voltage:

The solution is either to choose MOSFETs where Q3's gate-source threshold voltage is significantly smaller than that of Q2, or, as @tobalt suggests, replace Q3 with a bipolar junction transistor, which will begin to conduct when Vin reaches 0.6V or so:

simulate this circuit
I also replaced Q1 with a BJT, for the case where the INH signal is also derived from Vin, and would invoke the same problematic behavior.
The result is Q2's gate-source voltage being stifled as Vin reaches 600mV:
