It will become much clearer if you use a model for your battery: a perfect voltage source in series with a perfect resistor.
For simplicity (note: I don't think there's any charger on the market that works like that), I'll assume that your charger is a perfect voltage source.
Starting with a discharged battery, the charger will provide a voltage above the "perfect" battery voltage - but the voltage you'd measure "outside" the battery would match the charging voltage - the difference is the result of V=RI applied to the parasitic resistor. As charging completes, the "perfect" battery voltage increase. Since the charger voltage stays the same, the current lowers, so V=RI stays true on the resistor. When the battery voltage reaches the charging voltage, there isn't room for any voltage across the resistor... and charging stops.