Alright. I see you posted a picture of the setup:
Let's analyze it.
There is a squiggly ground alligator clip wire attached to the scope probe, therefore fast rise time measurements will be incorrect due to probe inductance. Therefore, I don't really trust the scope traces posted.
There seem to be long wires leading to an offscreen power supply, which hints at a high inductance in the power supply... but the tiny test PCB itself is too small to make out.
Now, I wonder if there is a local supply decoupling capacitor on this tiny board. There should also be a current limiting resistor in series with the LED.
Since none are mentioned in the description, I will go with the worst case: no resistor, and no capacitor.
- MOSFET turns on. It seems clean.
- LED current rises very slowly due to wiring inductance.
- MOSFET turns off, also cleanly.
At turn-off, interesting things happen. Since there is no local decoupling capacitor on the supply, once the FET is open, we have a series LC resonant tank including the wire inductance and the LED and FET capacitances. It rings, producing the huge oscillations at turn-off.
The ringing is bad enough to reverse bias the LED by more than 20 volts, which ensures a quick discharging of its capacitance, and therefore a quick turn-off. This explains why it turns off a lot faster than it turns on.
Also, if you look at the trace from your light detector (blue) you'll notice that the LED turns back on a tiny bit, when the ringing current forward biases it. But that could also be coupling between your sensor and your wiring, if it is close.
Now, the fact you post this scope trace, with such humongous ringing, and it doesn't seem to bother you, makes me think... that you kinda need help with high speed stuff.
Now... in order of complexity:
If your LED has Vf of a few volts, just drive it from a beefy and fast buffer like 74ACxx, with a resistor of course. Use a SMD part, over ground plane, with solid decoupling, clean layout, etc. For faster turn on and turn off, bypass the current limiting resistor with a small capacitor, equivalent to the LED capacitance.
As far as other methods go, if you want fast turn off, you also need a way to quickly discharge the LED capacitance when you turn it off (which the logic gate scheme provides for free). Your FET will not do this. It will switch the LED ON quick, but OFF will be slow.
Other ways to do it are complex enough... you should try the paralleled logic gates first.