Summary:
Avoid using a switched mode power supply (SMPS) if at all possible.
Use a high PSRR regulator either without a SMPS or following one.
Look at all the usual design advice on PCB design etc - see below and othes
Look at all the PSRR design references below
You say input is 2V - 4V. Why is this? What is the energy source?
If batteries,
1 x LiIon ~= 3V- 4.2V
1 x LiFePO4 ~= 2V - 3.5V
2 x NimH ~= 2V - 2.8V
...?
You mention "opamps". Are they 5V single supply ?
eg 2 x LiIon will give you 6V - 8.4V with most operation in the 6.5V - 7.5V range. Using a linear regulator gives you about 70% efficiency typically BUT a vast potential reduction in power supply noise. 5 x NiMh gives you say 5.x V to 6.5V and somewhat higher average efficiency.
If you MUST use a Switched Mode Power Supply (SMPS), which will add to your woes with its noise compared to a good quality purely linear supply, then after you follow all the good practices that everyone else will tell you about you are going to also want a superb linear regulator to follow the smps.
For the smps you'll
- Consider using a sinusoidal waveform or resonant design (Royer maybe).
- You'll want to look at a balance L-C-L input and output filter to the smps
- with balanced bifilar wound common mode filtering choke.
- All the usual care with return currents stuff,
- No loops in PCB go & return paths to pick up em noise
- No slots intersecting ground plane return paths
- Decoupling caps AT IC pins with possibly several caps of a range of values
- Ceramic caps and tantalum maybe (on output)(Tantalum on input OKish if utterly spike free operation above Cap_Vmax is guaranteed.
- With a smps pay attention to any oscillatory portions of waveforms, leakage inductance that isn't properly snubbed, diodes not to hard recovery, switching waveforms somewhat bandwidth limited, ... .
- A switched capacitor smps will have the advantage of no magnetic field interference - still has E field (minor) and power supply noise (can be significant.
- And ...
BUT
You will then want a linear regulator with an immensely good PSRR (Power Supply Rejection Ratio).
Ye olde standard LM317 claims up to around 80 dB PSRR BUT results can vary vastly across frequency range and with implementation.
If you search eg Digikey for PSRR you'll get devices that claim high PSRR as it doesn't get mentioned unless they are trying to make it better than usual. I got 206 voltage regulators with PSRR mentioned at Digikey - a small number compared to most searches.
As a disturbing example, (not enough current for your application but indicative of the PSRR performance you may expect) here's the datasheet for the TI TPS717xx which mentions low noise and "high bandwidth PSRR" in the title. BUT claoms 70/67/45 dB PSRR at 1/100/1000 kHz. That's actually good BUT that may not be obvious looking at apparent datasheet figures for standard parts. From the graph below it's clear that it would be a REALLY good idea to get rod of as much HF noise before the regulator as possible. This is reasonably easy using a mains sourced linear supply, and "not so easy" using a SMPS ahead of the high PSRR regulator.
I looked at this datasheet based on it being the dearest in 1000 quantity at Digikey that mentions PSRR
It's far worse than the TPS717xx - but much higher current rated.

Real world design advice:
Here's a useful Maxim application note on good PSRR design.
Here's an Omicron PSRR testing application note - they want to sell you test gear but it's a useful guide to what you are trying to achieve.
If serious or desperate enough this for $ paper may be of use 96dB rejection ratio PSU design.
Here's a useful TI design note) making the good point that performance may vary "strangely" across frequency.
As ever Wikipedia have something to say
Power supply
Output: 5V and 600mA. (Worst case, 200mA typical).
Supply: 2VDC - 3VDC (2 x Alkaline)
Assume 250 mA mean.
At say Vbattery = 2.5V and say 80% all up converter efficiency:
Iin = 5V x 0.25 A / 2.5V x 1/80% = 625 mA
A 2500 mAh capacity AA cell will last nominally 2.5/0.625 ~= 4 hours.
For NimH the voltage of 2 cells under that order of load will be 2.4V typical so slightly lower life nominal but they achieve closer to their rated capacity than Alkalines as loads start to approach 1C (here load = C/4).
Only the very very best AA NimH actually achieve 2500 mAh and then only when new so real world lifetime will be 4 hours or less.
Is a 4 hour battery life acceptable?
Can you increase battery voltage to allow a linear only supply?
What is the lowest opamp etc supply rail that you can really tolerate? (eg 5.0V, 4.9V, 4.5V ...?)
Overall real world result with NimH is liable to be better than using Alkalines.