A battery is simply a source of power or force.
You can think of an electric circuit like a locomotive pulling a train of carriages around a circular track, where the back of the train loops back to be coupled to the front of the locomotive.
When in a closed track the locomotive will use it's pulling force to pull the carriages around that track. How fast the train will turn is dictated by the resistance (friction) of the train.
In an electric circuit the battery produces an electro-motive force, EMF, voltage that pulls electrons round the circuit. How fast those electrons move, the current, is dictated by the resistance of the circuit.
How many carriages a locomotive can pull, or how fast it can go with a given load, is limited by the torque, pulling force, the locomotive can exert. If you want to want to pull a longer train, you need to add more locomotives.
Similarly, How much load you can drive, or how much current you can put through a given load, is limited by the EMF of the battery. If you want to want to pull more current, or drive the load harder, you need to add more batteries.
Of course, as you are no doubt aware, electrons go the opposite way, that is from negative to positive, but this model still works, just consider the train does not move but the track does instead, rotating backwards.
With two locos, when the track is complete, the first locomotive pulls the second one and that force is transferred through the second locomotive and adds to the force exerted by the second locomotive to generate double the force on the train of carriages.
When the track is broken, the locomotives can't move. They have zero speed.
The same goes for your two batteries, when the circuit is open, there is an infinite resistance (almost) and no current can flow.