You are partly right, in that electrons will move from a potential to a higher potential. So you may think that if I have a voltage at a point in space **A** with a wire running to a  different point in space **B** that electrons will flow through the wire.

<!-- Begin schematic: In order to preserve an editable schematic, please
     don't edit this section directly.
     Click the "edit" link below the image in the preview instead. -->

![schematic](https://i.sstatic.net/u9Uto.png)

<!-- End schematic -->

However voltage on its own means nothing. A voltage is a difference in potential between two points.

<!-- Begin schematic: In order to preserve an editable schematic, please
     don't edit this section directly.
     Click the "edit" link below the image in the preview instead. -->

![schematic](https://i.sstatic.net/2XVm1.png)

<!-- End schematic -->

That is you need a common reference point in order to actually assert the voltages in the first place. You therefore end up with a loop, with some resistance, whether you like it or not.

However, that also brings up another point. In the circuit above, even without the reference wire, there is still some resistance between the negative sides of the voltage sources, all be it huge. As such you have a loop, and a tiny current will flow in the wire, though it may be too small for you to measure it.

It is also important to separate the notion of current to the notion of electron movement. Current is an abstract representation whereas electron movement is a physical phenomenon. We say current loops through the battery or capacitor, but in actuality, the electrons do not. Rather, an equal number of electrons exit them as do enter the other side. That difference is subtle, but important.