It looks like LiPo batteries stay above 3.3V in normal charge levels, and drop to 3.0V when they're below ~80% capacity.
Image source: https://learn.adafruit.com/li-ion-and-lipoly-batteries/voltages
Based on this, you can stick with an LDO since it's the cheapest and easiest way to step down a voltage, assuming you don't mind the efficiency losses when your battery is >4.0V. LDOs will dissipate any unused power as heat; the greater the difference between Vin and Vout, the lower the efficiency. A benefit when compared with switching topologies is a much smaller ripple on Vout, which is good for sensitive electronics.