If we consider only ideal components, then the only number you need to care about is maximum stored energy per unit cost. As you already found, the energy stored in a capacitor is given by
$$ 1/2 \cdot C V^2 $$
The SAMWHA supercapacitor can thus store a maximum of:
$$ 1/2 \cdot 500 \cdot 2.7^2 = 1823 \:\mathrm J $$
The large electrolytic can store up to:
$$ 1/2 \cdot 0.0022 \cdot 450^2 = 222.75 \:\mathrm J $$
In this perspective, only the energy storage matters. That can come from high capacitance or high maximum voltage. It doesn't matter which, because you can shift the balance between current and voltage by changing the design of your coil. A coil with a large number of turns will develop a strong magnetic field without much current. However, it will have a large inductance and will require a high voltage to develop that current quickly. A coil with fewer turns will require more current to develop the same magnetic field, but the inductance will be lower so less voltage is required to get the voltage to rise rapidly.
If you were purchasing ideal capacitors, and you could fabricate an ideal coil, then this would be all you needed to make your decision. However, the real world is not so simple.
Real capacitors have non-ideal effects. It's common to model a capacitor as a network of other components, like this:
simulate this circuit – Schematic created using CircuitLab
Leakage determines how quickly the capacitor self-discharges. Ideally, this value is infinitely high. It is probably not much of an issue in your application, since you are not storing a charge for a long time.
ESL is the equivalent series inductance. Ideally this value is 0. This inductance places an upper bound on how quickly the current through the capacitor can change. You will want to make sure that the combined ESL of all your capacitors is at least an order of magnitude lower than the inductance of your coil, otherwise you will not get current, and thus the magnetic field, to rise quickly.
ESR is the equivalent series resistance. Ideally this value is 0. Here is the real problem for your application. You will be, at peak, drawing a very high current through your capacitors. This current must also flow through the ESR, and it is subject to all the physical laws of resistance. That includes dropping voltage, according to Ohm's law:
$$ E = IR $$
And converting electrical energy to heat, according to Joule heating:
$$ P = I^2 R $$
Any voltage dropped across ESR is voltage not available to drive your coil. If current is high enough relative to the ESR, then the voltage your coil sees will be essentially nothing. What you have is essentially this:
simulate this circuit
For the current to increase, voltage across R1 must increase. Since the voltage across R1 plus the voltage across L1 must equal the capacitor's voltage, as the current increases, the voltage across L1 must decrease. That's bad for you, because it means you can ramp up the current in L1 less rapidly.
Furthermore, the heating caused by losses in the capacitor can damage the capacitor. That's also bad for you.
The capacitor datasheets will elaborate on these details. However, based on general characteristics of supercapacitors and electrolytics, I can tell you that the electrolytics will have a much lower ESR on average. Supercapacitors are intended for lower current applications, sitting somewhere between a battery and a capacitor. They will typically be damaged by high current, or at least their ESR will be so high they will not perform well in your application.
There are also particular electrolytic capacitors designed to have an especially low ESR. These are called simply "low ESR" capacitors. They are also more expensive, but you might look into them. They typically find application as ripple filters in power supplies, where a higher ESR in the constant charge-discharging cycle they do would be a problem for efficiency, performance, or reliability.