If we guarantee that the terminal voltage will drop to some value
under 4.2V, is it safe to connect the battery to the source?
But how can you guarantee that the battery voltage will not exceed 4.2 V? Electrically a Lithium-ion battery looks like a large capacitor, and just like a capacitor if you push even a tiny current into it the voltage will continue to rise as it accumulates charge. Any amount of current that charges the battery at all will eventually overcharge it.
Therefore you need an external method of preventing the voltage from rising above 4.2 V. A crude way of doing it is to simply measure the cell voltage (either with a comparator circuit or a multimeter), and electronically or manually cut the charging current when it reaches 4.2 V. But this will only get to around 70~80% of capacity, depending on the charge rate.
A more sophisticated charging circuit would automatically reduce the charge current if voltage tries to go above 4.2 V. This is called CVCC (Constant Voltage Constant Current) charging. It isn't good for the cell long-term though, because it keeps it at maximum stress. Most Li-ion chargers shut off completely when current drops to ~10% of the constant charging current, which gets to ~99% of maximum capacity.
Many chargers also have a timeout and/or capacity limit as a backup in case the CVCC cycle fails to complete, and a lower charge rate if the cell voltage is very low. Finally the cell temperature can be monitored to (hopefully) cut the charge off before it explodes if something goes wrong. As well as all that, a PCM (Protection Circuit Module) may be built into the battery or on the charger, to cut it off if the voltage goes too high or low.
These extra safety functions are required for consumer products that may be left unattended in situations where an exploding battery would be disastrous. If you need to charge a Li-ion cell and all you have is a constant current then if you constantly monitor the cell voltage, and if you don't try to charge an over-discharged or damaged cell, and if you keep it away from flammable material and have a fire extinguisher nearby then it should be OK. But you must guarantee that the battery limits won't be exceeded.