High frequency noise currents flowing in the shielded conductor capacitively couple to the shield and use the shield as a return wire. If you leave the shield floating, it must flow through the capactive coupling between shield and ground to circulate and complete the current loop. The electromagnetic energy energy traveling through the air via the capacitive coupling is the EMI itself.
But if you ground the shield properly so that flowing in the solid conductors is an easier path (lower impedance) than through the capacitive coupling in the air, the electromagnetic energy stays in the solid conductors and out of the air resulting in less radiated EMI.
Similarly, if an external EM wave induces noise currents to flow in an ungrounded shield, they will capacitively couple into the cable and use the cable to flow to ground. This manifests as noise on the cable. But if you ground the shield, then the induced currents in the shield stay in the shield, and out of the shielded conductor, on their way to ground.
Note in the diagrams below that these are high frequency currents subject to the skin effect so they cannot travel through the shield. They must travel to the end of the shield, and around the edge in order to get to the shield's outer surface.
Also note that if you provide a grounding connection that is low resistance, but high impedance, the noise currents won't flow through it since the impedance dominates. So even if you ground the shield with a piece of wire of low resistance, the high frequency noise currents may still not flow through if the impedance through the wire is higher than the impedance of capacitively coupling through the air. That's why shields should have a continuous connection to ground along their entire open edge. These are high frequencies subject to the skin effect so they care more about distance they need to travel and perimeter of connecting surfaces more than the "bulk" of conductive material in the connection.

From EMC Engineering, Henry Ott 2009