Okay, I've been chasing this down in my head for months now. I've built a couple prototypes, as an exercise in understanding the fields involved. I finally have an answer I can believe.
Say you have the original concept, a capacitor inside a capacitor. Compare that to this:

I would argue that this circuit is identical to our four-plate arrangement. Each of the inner plates of our four-plate stack is still a conductor with a great deal of surface area, and large capacitance to the plates on either side. We've drawn them as two separate plates with no impedance between them, but that electrically changes nothing. Now the circuit looks more familiar. It's really just three capacitors. And the one across the secondary really doesn't add anything, it just creates a voltage divider. You'll get that when you attach a load anyway.
This has some very similar properties to a transformer. DC can't pass from primary to secondary, but AC can. This makes the system galvanically isolated. However, this does not necessarily make it isolated for practical purposes! If you put AC between the primary and secondary of an ideal transformer, nothing happens. If you put AC between the primary and secondary of this circuit, you get lots of current flow. So this would fail an AC hi-pot test, and common-mode noise on one side would transfer happily to the other.
If those aren't problems for an application, there may be some advantages to this over a magnetic transformer. For one, you can transfer more power at higher frequencies, somewhat the inverse of a transformer. (Depending on the transformer, of course.) There are no obscurities of core materials and geometries to deal with. I suspect it's more efficient than a transformer, though I have no data to demonstrate that. Instead of eddy currents, hysteresis losses, and winding losses, all we have is the ESR loss in the capacitors, which I'd expect to be much lower. And it's DC-safe! If you put DC on a transformer, the core saturates and you probably break something. Put DC on this, and absolutely nothing happens.
Now, why can't we step up, if it's truly the dual of a transformer? Because electric fields and magnetic fields have some fundamental asymmetries. An electric field starts on a positive charge and ends on a negative charge. You can't expose a conductor to another conductor's electric field; the electric field of a capacitor definitionally involves two conductors, and if you try to introduce a third, it just moves some of the termination points. (Cartoon version, I am not a physicist.) But a magnetic field always ends where it begins, so a single conductor can have a magnetic field that the secondary can be exposed to with varying geometry.
In other words, it's because electric fields are unipolar, with each end on a separate particle. Magnetic fields are dipolar, starting and ending on opposite poles of the same magnet, forming loops. So amusingly, @JustJeff 's comment was inverted! We really need an electric dipole, not a magnetic monopole!
If a transformer is two conductors sharing a magnetic field, its dual would be two conductors sharing an electric field. In other words, the dual of the transformer is a pair of capacitors.