I'm trying to understand the physics. The spark plug has a gap of 0.9mm. Why would the current go through the insulation resistance, the finger resistance and the air gap between the finger and the engine ground instead of traveling a much less resistance through the 0.9mm gap?
Your finger on the wire forms a capacitor to the plug wire center conductor; the wire sheath is the dielectric. This couples to your body, forming a DC path to your torso. So the AC-coupled high-tension charge is flowing in and out of your finger to your body.
How? Even your ungrounded body presents a non-trivial capacitance to ground through the surrounding air, and in between any insulator and car ground. There will be some current, like static electricity.
Now if you’re leaning on the fender, that makes a fairly big capacitance to car body ground. Given the voltages involved (~50kV for some systems), while you may not see a spark on your finger, you still might feel some current. Stand away from the fender and the body-to-car capacitance - and sensation you feel - will be less.
This across-the-insulator capacitive coupling between the plug lead and ground is why plug wires are routed away from the engine on insulating standoffs: to reduce shunt losses in the lead and instead deliver all the coil’s energy to the plug where you want it.
The underlying physics is called displacement current, which is a fundamental part of how capacitors work (electric fields straining electron orbits). A change in charge induced by voltage change is accompanied by a current into and out of the capacitor as the charge changes. There doesn’t need to be spark for that to happen, current still flows.