(Edit: I just realized in your first image that you don't have the drain of the MOSFET connected to anything. In which case it isn't the body diode doing this. In that case, the MOSFET body is being coupled to the environment and picking up some small current. See the second part of my answer. I'm going to leave the rest of my answer here because I think it might be useful to you)
Every MOSFET has a body diode. This is a parasitic element and is a result of the way that MOSFETs are created at the silicon level. We can't do anything about them so we have to design around it. In a PMOS, the cathode of this diode is connected to the drain and the anode is connected to the source. So, what this means, is that with your current configuration, current can flow through the body diode at all times since the diode is forward biased. You need to swap the high side of the power supply to be connected to the source, with the led connected to the drain.
Additionally, don't leave MOSFET gates floating. It takes an extremely small amount of charge to turn a MOSFET on and so if it's floating, it can easily turn on due to static charges or being coupled to other things in the environment. You should add a high valued resistor from the gate to the positive side of your power supply.
But this doesn't explain why you can touch the MOSFET to the led and have it turn on or why it turns on with nothing connected to the drain side of the MOSFET. This is most likely an entirely different phenomenon. LEDs are super efficient and can turn on with miniscule current. Try this: Add another jumper wire to your circuit on the negative side of the led. Pinch the other end of the jumper wire pin in your fingers so you are connected to the negative side of the led. Now try touching the MOSFET back to the led. I think this will prevent it from lighting up. Before, AC power from mains voltages was coupling through your body and going to ground through the LED or coupling through the MOSFET body. This isn't dangerous because the current is tiny and it happens all the time, we just don't notice it. The LED is a nice indication of this fact.
And if you've only connected things as described here, then you most likely haven't damaged anything. Body diodes in MOSFETs are not designed to be continuous paths for large amounts of currents, but can typically handle small loads.