I'd suggest your issue is the path for the return currents.
You might like to think of just the signal wires, but for every single-ended signal you drive, there has to be a return path.
If there are no low impedance ground wires near by (remembering that impedance here is tied to the loop area between your signal and return wire), you will tend to get significant cross-coupling of signals. The large loops will also be perfect for coupling external RFI.
There are really just three solutions, all involve reducing loop areas:
- Have lots of ground wires through the width of the cable, ideally alternating signal and ground pins (the classic setup for old IDE cables)
- Use differential drivers for your cable links, with the +/- signals on adjacent pins (converting to differential for the link, then back again at the other end)
- Have an parallel conductor (like your foil) which is connected to ground at both ends, ideally at multiple points, through a custom connector.
The reason why the last option requires a custom connector, is that most user of IDC cables opt for one of the first two options...
If you really can't change your overall pin assignment, I'd suggest you make your own custom IDC cable with an integrated ground plane. This is fairly easy to do using self-adhesive copper tape running length wise on one side of the ribbon cable. At each end solder on a couple of short wires with single socket 2.54mm connectors.
Tweak your design to include suitably located single pin headers connected directly to your PCB ground planes, and when you insert your cables into the IDC header, use these pins to connect the ground fly-wires.
I strongly suspect this will give you much better signal integrity, as the loop area for each signal is now very significantly reduced...
You might even be able to get away with just one wire to the ground strip, located in the middle at each end, as this still greatly reduces the loop area. And instead of adding header pins, you could trying soldering a short flying wire to either your top ground plane (if you have one) of the negative of a near by decoupling cap.
If you do use differential signals or alternating signal-ground, the "ultimate" ribbon cable is "Spectra Strip" which twists together the signal and ground pairs for most of the length, with regular untwisted lengths for adding IDC connectors. Great stuff when you need extra integrity, but a little expensive.
