The highest performance (most power efficient/coolest) method is to use a FET OR-ing setup. Their primary advantage is a near-zero voltage drop, limited only by the RDS(on) of the FET and current sense resistor (10 mΩ total resistance is fairly easy, but 1-2 mΩ if you really need).
Controllers for said systems typically use a low-value sense resistor and appropriately-sized FET to connect the power supply to the circuit. It measures the voltage across the sense resistor to ensure current is flowing in to the device from its supply, rather than being siphoned off from the other supply, then the FET does the switching. Some controllers don't use a sense resistor and just measure across the FET.
Here's an article on Digi-Key's site about ORing controllers.
Furthermore, if your system uses a mid- to low-current (<5 A), there are even controllers with integrated FETs, so your component count can be super low. But if you're in the low-current range and if you can get away with the voltage drop of some ORing diodes (0.3 V for some Schottkys), you're not going to beat them on price.
Anyways, several suppliers make these controllers (search for "power supply oring controller" on Google/DK/Mouser/etc.), to name a few: TI, Linear, Micrel, IR, Maxim, and Vicor.