There are many sources of loss in a SMPS. Here are just two:
Each time the "switch" (usually a MOSFET) changes from on to off, it spends some time in a state between. During this time, there is simultaneously significant current in the device, and significant voltage across it. As \$P=IE\$, this means the MOSFET is wasting input electrical energy to warm itself.
To minimize these losses, you want to switch as little as possible. The lower the frequency, the less often you switch, and the lower the switching losses.
resistive losses in the inductor
A real inductor has resistance, and will experience losses according to Joule's law: \$P=I^2 R\$. The current in this inductor is not constant: it grows when it is storing input energy, and it decreases as it is releasing that stored energy to the output.
Clearly, the point of the SMPS is to deliver current to the load, so we can not eliminate this current. However, we do have a choice of the shape of that current. The output filter (often as simple as a capacitor) means we can have high current ripple in the inductor and still get a steady output voltage. However, if there is high current ripple, there must necessarily be times where the inductor current is much more than the current required by the load. Since losses are proportional to the square of current, this means our losses are higher when there is current ripple, compared to the case when there is no current ripple.
To minimize these losses, you want the switching frequency as high as possible, as this will minimize the current ripple.
If the "best" frequency is one that has minimal losses, then the best switching frequency is the optimal balance between multiple kinds of loss. These are not the only kinds of loss. To determine the best frequency, you will have to perform a careful analysis of all the potential losses in your specific SMPS, determine how those losses change as frequency, then solve that system of equations for a minimum loss.