I don't have much practical knowledge of electronics. I am trying to operate esp12e with single Li-ion battery with charging circuit. I am planning to use a good LDO regulator because Li-ion battery voltage can get high upto 4.2 volts while we need 3.2-3.6 Volt for esp. My problem is when the battery voltage is dropped below the certain level (near 3.4 volts), the regulator will not be able to further drop the voltage and stop.At this point,I want to use the battery directly, without the regulator. How can I achieve it? Please help.
You can do even better if you get a buck-boost regulator like the one below: -
Basically it produces 3.3 volts from an input voltage of 2.5 volts to 4.2 volts.
Note - irrespective of the regulator technique you still must use a Li-ion battery protection circuit to prevent over discharge of the cell.
Modern CMOS LDO regulators will more-or-less do what you describe. They need a certain voltage to start conducting, for voltages between that and a bit above the output regulated voltage they will pass the input voltage minus a (typically small) voltage drop.
For example, the Rohm BH33MA3WHFV-TR in the BHxxMA3 series.
As you can see, above about 1.5V typically you get essentially the input voltage at the output until regulation kicks in (at 3V in this example plot).
The 1.5V is a typical figure however it's well below the minimum voltage you should allow the battery to discharge to (maybe 2.8 to 3.0V) in order to prevent permanent damage to the battery.
Avoid older bipolar designs- as well as some having high dropout voltage, some of them use low hFE lateral PNP pass transistors which means the regulator internal current draw increases drastically near drop-out.