Buck converters are as simple as boost converters. In fact, they are exactly the same circuit, just seen backwards, if we have the freedom to choose which switch (out of the two) will work as the controlled switch (or both, if it is a synchronous converter).
Regarding your second paragraph, if you did that, you would incur in losses. More than with an inductor-based switched regulator, and much much more than with a linear regulator. Every time you connect a voltage source to a capacitor whose initial voltage is not the same as that of the voltage source, you unavoidably waste energy. Even if you don't see an explicit resistor, in real life it is there, and (curiously) no matter how small it is, it will waste that same amount of energy. See here.
Charge pumps work as you say, but they are less efficient than inductor-based switched regulators.
So, that's the justification for the --apparently unnecessary-- added complexity of inductor-based switched regulators.
More: To try to give you the intuition of why buck and boost converters exist, see this figure.
If you try to move energy between two voltage sources that are not alike, or between two current sources that are not alike, you will have unavoidable losses. On the other hand, you can move energy (and even doing some voltage or current scaling on the way) without any loss, if you connect a voltage source to a current source. The passive physical element that resembles the most a current source is an inductor. That's why inductor-based switched regulators exist.
Charge pumps would be on the left column. Their theoretical maximum efficiency is lower than 100% (the actual efficiency depends on the difference of voltages, and the capacitances). Inductor-based switched regulators are on the right column. Their theoretical maximum efficiency is 100% (!).