No, the battery does constantly not give 3.7V. This is the voltage value at a way lower capacity.
3.7V does not means much. That is the value at which the battery is most stable at, but the actual value when fully charged is 4.2V, so a charger will have to provide higher than this if you want to fully charge it.
At 3.3V standard Li-Ion cells are considered discharged because they can no longer provide enough sustained current for the average applications they were designed for.
The minimum voltage for charging a standard Li-Ion is 4.201V. But considering impedances of the charger and cell, most chargers have 4.25 or even 4.3V when running blank (not connected to a cell).
Although those values have been chosen this way they are not like the 10 commandments. You can discharge a cell under 3.3V but it will provide less current and you can charge it even up to 4.3+V and it will have higher capacity, but both practices will lower its lifetime.
So in emergency cases, overcharge them to 4.3V and you'll have some extra capacity to work with and if you want to have a very long time for a cell, let it charge only up to 4V but that will only provide ~80% of the stated capacity.
Getting back to chargers, I'd design my charger like this:
Option 1: 4.0V - safe mode - for safe charging and long life and overdischarged/bad cell recovery.
Option 2: 4.21V - standard - normal charge mode (bad cells with reduced capacity will overheat if charged at standard level)
Option 3: 4.32V - overpower - over-charge mode (lower cell life, very high danger for reduced capacity cells)