Do not connect the battery directly if you want this ESP8266 to last. 3.7V is the nominal voltage but it can be a little higher when they are charged. You can place an ld1117 but even though they are marked as LDO (low drop out regulator) they do have some significant dropout, over 1V. And since Li-ion batteries have a voltage curve (their voltage drops when discharged) you will end up with a voltage close to the minimum for the ESP8266.
I wouldn't recommend using a standard ld1117. Find another regulator with a much lower dropout but make sure it can handle 300mA since the ESP8266 has 300mA peaks (I've measured) when transmitting at full power.
Or better yet, use a switching regulador (step-down), they are cheap and ultra easy to use. You can search in mouser.com, there are many easy to use regulators, you can even use the classic MC34063A.
Or another thing you can do, which I've done some times, is placing a schottky diode in series with the battery, the diode will drop around 0.4V so you will end up with 3.3-3.4 when the battery is charged, and drop down to 2.8 when it's almost empty.
On your last question, most Li-ion batteries come with a protection built in that avoid over discharge damage and short circuit damage. Just make sure your battery has that little pcb (most do). So you don't have to do anything special, when the battery is very low, this protection circuit will disconnect it from your circuit to avoid damage.