This is a classical case for a step-down / buck converter.
Yes, switching something generates noise, but as everything, we can determine that and technically limit it to a level below significance.
- Use a modern step down converter. TI's SimpleSwitcher modules are probably a good, no-worries, everything included package.
- Make sure the switching frequency is well above what the signals relevant to your device are designed to pick up
- Use appropriate noise rejection: Proper decoupling capacitors (as you'd need them on a linear supply, too), and ferrite beads designed to get rid of higher frequencies are probably sufficient to deal with the noise that still comes out of well-designed switch-mode regulators (which is, in fact, relatively well-controlled!)
- Since you probably don't want your bluetooth device to directly couple into your most sensitive amplifier circuitry, you'd probably move it furthest away from the input signal section, anyway. That's a good place for a power supply, too. Generally: close to the device's main power supply should work!
I'm a lazy person. I simply went to the TI website, clicked myself through to the power modules (step-down), sorted by price, selected the cheapest module that would do my job (26–42V in, 3–5V out, 0.5A), and then clicked on the "interactive design" button. That led me to another option, which would require me to buy an inductor, but would be cheaper.
If you're really worried about noise:
You often see circuitry that first steps down using a switch-mode buck converter for the "large" voltage difference, and then does the remaining 0.5 V to 1.5 V difference with a LDO (low dropout) regulator. You'll find that in multi-supply RF machinery, for example, where noise is a lot more important than here.
However, you're, in the end, driving a device with that that probably has some internal means to deal with small voltage deviations as well as it's digital anyway, so I think you might be overthinking this – simply go for the switch-mode supply.