This question is a bit broad to belong here, but generally there are three classes of approach:
1) Use a software solution to put the chip into the lowest power sleep mode possible after it is done operation. Have a button that resets it and/or wakes it from sleep. Possibly replace the linear regulator with one having a lower quiescent current
2) Rework the board replacing the SOT23-5 linear regulator with one of the many having an active high enable input on pin 4. Connect the enable input to Vin via the button, and also to a GPIO via a diode. Push the button to start the system, software then holds the GPIO high to keep the system on as long as it needs to, then turns itself off. Depending on startup time you might need a capacitor to help hold it on long enough after the button push for the software to take over holding the regulator enabled. (Some toys actually use a capacitor from Vin to the enable to auto-power on when the battery is connected, and then power off when software decides they've been idle too long)
3) Place a non-inverting power switch chip chosen for low quiescent current in the upstream supply path. It's important that this be a non-inverting configuration, ie, active high enable for a high side switch. An inverting switch such as a single P-FET or N-FET will just end up turning itself partially back on through the protection diodes and pulling resistor.
Make sure to use a power source which can satisfy the peak current demand. A CR2032 coin cell for example may satisfy the average power requirement but won't meet the peak requirement and the ESP will be unable to boot. Also be very careful not to have any I/Os asserted against pulling resistors for meaningful amounts of time - especially on an ESP where you have lots of pulling resistor to configure boot mode.