If you want to modify an exsiting circuit, things will get a bit more complicated.
In that case, your first idea - using a switch to basically swap out the LiPo cell - is probably best.
The reason is simple: a battery charger is designed for the cell type it is charging. and for the voltage of the entire battery.
LiPo are a bit tricky because they have this tendency to combust; the youtube videos of LiPos bursting into flames are real!
When modding and exsisting charger with a switch, make sure the switch is a break-before-make type.
This means that when rotating, there is a point where all contacts are open before any contact is closed. This will avoid having two cells connected in parallel.
If you connect 2 cells in paralel, one charged, one discharged, the discharged cell will draw current from the full cell.
This seems no big deal, but the current flowing into the discharged cell is not regulated or limited - and that can harm the cell: LiPo charging must be done with a limited current.
This would happen every time you switch over with a make-before-break type switch.
If you use a cell type different from the original cell type in the power bank, make sure it is a cell with larger capacity.
LiPo chargers limit the current they feed to the cell. The current limit is defined by the cell capacity.
"too little" is harmless while "too much" damages the cell; so the current limit designed for a say 1aH cell is usually harmless for a 1.8aH cell.
"usually" not "always" because there are cells designed to allow for high charge current and others not so much; so the limit for a 1Ah quickcharge cell could be damaging for a 2aH "standard" type.
So your original schematic was very much on the right track: all negative terminals wired together and to the ground of the whole contraption, the positive terminals of the cells feed a selector switch and that feeds the positive input of the charger board.
If all cells are of the type the charger is designed for, this will work; from the chargers point of view, you are simply swapping out the battery cell.
To increase the life of your cells, recharge them soon after discharge: the cells self-discharge then has little chance to deep-discharge the cell (which damages the cell).
Read up on LiPo batteries on the web.
For example: http://linrunner.de/en/tlp/docs/tlp-faq.html#battery (considering laptop batteries but follow the lead from there).
If you still dare :-) go for the switch.
In a recent project (an art installation) I needed some battery: the power was fed to a small vehicle hanging from a rail; this used a sliding contact and those graphite-on-brass contatcs get wonky over time (the brass oxidizes). So I wanted a battery in the vehicle to avoid power interruptions to the onboard computer.
I chose lead, one of the oldest battery chemistries out there :-)
(actually in the more modern form with an absorband glass mat to bind the electrolyte).
Lead does not have the best energy-to-weight or energy-to-volume ratio (actually: it has the worst). But they are really safe...