The regulator repeatedly pulses current through L1, delivering energy to it, then stands off while that energy is transferred to C2. The regulator does this as often as necessary to keep C2 "topped up" to whatever output voltage is desired.
These transfers of energy from L1 to C2 are very small, but very frequent, anywhere from tens of thousands of "packets per second" to millions. Contrast this frequency with linear supplies, using bridge rectifiers; the capacitors you find in them have to be much larger, because they are required to maintain a reasonably steady voltage for much longer, "top-up" pulses from the rectifier happening only 100 or 120 times per second.
The input capacitor C1 in the switching supply you showed is to "smooth" voltage at the regulator's input, by acting as a short-term source of current as the load's demands change. However, because current is being drawn from that source in short, frequent pulses, current demand is very short-term, and only a small capacitance is required.
You find small capacitors across the power supplies of many elements in a circuit. There are usually long wires and PCB traces between a power supply and the elements being powered, and those paths have inductance which prevents current in them from changing rapidly. Therefore, any element that switches something rapidly on or off will find that the power supply simply can't keep up with changes in current demand. The result is dips and peaks in voltage at the supply rails, which is bad for two reasons:
- The element is messing up its own supply voltage
- the element is messing up everybody else's supply voltage
Small capacitors across the supply near each element act as a short-term source of energy, able to respond to that element's fast-changing current demands. This helps mitigate the dips and peaks in supply voltage for the element in question and any nearby potential victims.