Because you're frustrated, here is the solution first thing! :)
Switch the 1V going into the top of the potentiometer, instead of the output connection.
In other words, use the potentiometer to modulate the amplitude of a 2KHz 1V square wave going into the op amp's noninverting terminal. Do not open and close this terminal, 0V means at ground potential. Open circuit is not 0V. But, as long as there is a potentiometer with one leg connected to ground, it will not be open circuit and work fine.
Note: the 33nF capacitor is to critically damp the LM324, but the compensation strategy will differ for each op amp, or often not even be needed if the op amp is internally compensated for unity gain.
Now for those interested, here is a somewhat lengthy explanation of what is going on, why there are current spikes, and why this change eliminates them.
Oh, actually, a quick note: you shouldn't refer to a gate as a base. One is a current actuated diode, the other acts like a charge actuated voltage-dependent capacitance. BJTs have bases, MOSFETs and IGBTs have gates. You called it by both terms in the question, just something to watch out for.
There will always be a spike, because electromagnetic waves take time to propagate at the very least (assuming you had some magical op amp of the infinite that uses unicorn farts instead of the standard blue smoke they normally have inside). But the far bigger problem with non-magic op amps like the one in your circuit is that they have a reaction time. Every control loop has reaction time. For there not to be a spike, your op amp would actually need to predict the future and begin slewing downward before the switch is closed to account for this reaction time (which is really just what we call 'phase lag' in the context of AC amplification. For a DC control loop, I prefer to think of it as reaction time but it's really just metaphorical sugar).
What you're doing is essentially causing a transient. This is no different from if you were physically connecting an LED to a current limited power supply, and taping the lead against one of the output terminals, making the LED flash on and off. It's like you're disconnecting and connecting the load physically. Indeed, that's exactly what you're doing, only its the output that you are disconnecting, but the point is that you're creating a discontinuity by adding and removing a load. The rise time is nanoseconds if not less.
Take any bench power supply with current regulation as well as voltage, set it to 2mA, set the voltage higher than the LED's drop, put a current sense resistor in series and view it on an oscilloscope, then hook up an LED. I promise you that any power supply, from a $50 sketch box to a $10,000 Agilent/Keyshite power supply will have a current spike just like you're seeing in your circuit. It will be lower or higher in amplitude and wider or narrower in time depending on the quality of the power supply, but it will always be present. Always. It's unavoidable. Your options are limited to acceptance or denial. Either way, the current spike will happen.
This is actually one of the main factors that separates a good power supply from a bad one, transient response. This is how fast it can respond to a sudden change like the one you're causing to happen at 2kHz. Nothing responds instantly, unless the datasheet has ∞V/µs as the slew rate for the op amp (it doesn't). A control loop for current is always fully on if there is no current flowing, because it wants 2mA not 0mA to be flowing. It doesn't matter if it is because there is no load for current to flow through, or no connection from the output to the pass transistor, it's ultimately the same thing - way less current than 2mA. A nice $1000 power supply will have very good transient response, on the order of 15µs for a 50% load step. That means if it is a 1A power supply, and you have it set to 500mA, and it is open circuit, then you connect a dead short, more than your 500mA current limit will flow into the load for only 15µs before going into regulation. Likewise, if the load suddenly doubles in resistance, the current going through it will fall beneath 500mA for 15µs before catching up. This current spike isn't a spike, its just the rise time and fall time of your control loop. And all control loops have them. You can lower the amplitude by using a resistor to limit the maximum possible current, and you can shorten it by making your control loop faster (which is also a trade off with stability), but you can't make it go away.
Plenty of power supplies will actually blow an LED if you turn the voltage all the way up but set the current to 2 ma. Try it. They'll dump enough current through the LED long enough to kill it before the control loop can get the current into regulation. Any time you have a load transient, there is going to be to much, or too little current going through it. Normally its brief enough that the device doesn't care. But it is there, even if you don't realize it.
Hopefully you understand why the spikes are occurring now. Since a load transient causes spikes that cannot be avoided, the solution is obvious - remove the load transient.
If the LED and control loop is always connected, and it is the noninverting input that is slewing from 0 to , then the current regulation point will follow that. There will be lag still, but the lag will be a slight delay in the wave form (it will be slightly out of phase) but it will still match. To return to the power supply metaphor, this is like you're leaving an LED connected to a power supply, but wiggling the current knob up and down really fast, in contrast to physically connecting and disconnecting the LED.