These are indeed the advantages of the switch-mode power supply:
You'll find that wall-warts also contain a transformer – but it's not the kind of transformer where the primary side directly plugs into the grid, and the secondary side directly delivers the voltage you need. Instead, you'll find flyback transformers, which basically switch either side (typically, the secondary) of that transformer very quickly, to extract just as much power from the magnetic field in the core to keep the output voltage stable.
Switching fast means the transformer can stay small. The amount of energy per "switching cycle" is very small, so the currents on the primary side are small, too. There's also no very large output caps to convert the rectified, but still 50 or 60Hz of the grid to low-ripple DC – the ripple is much much much higher in frequency, and can be filtered out with much smaller components, too.
I want to make a power supply with the current capacity of one of these wall warts so I'm curious.
Be a bit careful, sounds like you want to plug something into grid voltage. Both wall-warts and traditional transformers offer isolation, so that you don't stand a chance to ever touch something that is directly connected to 220 V (or 120 V, or whatever your country has). You must adhere to that design objective!
You'll have a hard time building something as compact, cheap, reliable and low-noise as commercial power supplies; these are designed by companies with decades in experience in both building safe and cheap supplies in million quantities.
Now, if you just need a supply to go from e.g. a 24V battery net to 5V, you can indeed have a lot of fun designing easy step-down converters (and given sufficient fuses, you don't even have to stick with isolated designs). There's a lot of controller ICs that you can freely choose from (and suppliers like TI have nice tools to design reference circuits for you), but you can also take a dive into the theory and design your own control loops :) (If you need it to just work: of course, just buy the IC and implement the reference design.)