There are plenty of cheap and cheerful (or nasty maybe?) circuits on eBay or AliExpress if you search for a "USB Boost Circuit" or similar. These sell as a finished device for $0.30 - $2 depending on how many you want, are usually around 25mmx19mm, and if you look carefully at the photos you can sometimes read the control IC product code and work out what it is (although better yet some AliExpress suppliers will provide the datasheet in the listing). Pretty much every one of these will have a resistor divider which scales the output voltage to a reference voltage and changing the resistor on one leg of divider will let you get 3V3 instead of the 5V they ship as.
I bought 50 to get the price down, desoldered the USB sockets, soldered on double AA holders (at $0.19), and now I have a stash of cheap power supplies that I can set to whatever voltage I need for a project. I don't think you can really beat $0.50 per power supply.
ETA: You shouldn't need the datasheet to find the voltage divider or calculate the new values for it, it should be obvious which is the divider (i.e. a pair of resistors from one IC pin, one resistor to ground one to output voltage) and the reference voltage they are targetting can be calculated from the existing divider and the 5V output currently set. On mine for example I have a \$15k\Omega\$ to ground and \$47k\Omega\$ to output so I have a reference voltage of \$\frac{15}{47+15}\times5V \approx 1.2V \$, and thus to get 3V3 output we can (by happy coincidence) swap either resistor for a \$27k\Omega\$ depending on whether you want to be marginally higher or lower than 3V3.