There are plenty of cheap and cheerful (or nasty maybe?) circuits on eBay or AliExpress if you search for a "USB Boost Circuit" or similar. These sell as a finished device for $0.30 - $2 depending on how many you want, are usually around 25mmx19mm, and if you look carefully at the photos you can sometimes read the control IC product code and work out what it is (although better yet some AliExpress suppliers will provide the datasheet in the listing). Pretty much every one of these will have a resistor divider which scales the output voltage to a reference voltage and changing the resistor on one leg of divider will let you get 3V3 instead of the 5V they ship as.
I bought 50 to get the price down, desoldered the USB sockets, soldered on double AA holders (at $0.19), and now I have a stash of cheap power supplies that I can set to whatever voltage I need for a project. I don't think you can really beat $0.50 per power supply.