A higher-valued inductor will hold more energy with a given amount of current flowing through it than would a smaller-valued inductor, but the rate at which current flows through the inductor will increase or decrease more slowly than with a smaller one. The rate at which energy flows into an inductor is proportional to the applied voltage and current being pushed through it. If a large inductor doesn't have much current flowing through it (e.g. because the switcher has a very light load), there is a limit to how quickly the switcher can "ramp up" to start delivering more current. Additionally, if a large inductor does have a lot of current flowing through it, that current represents energy that's going to have to go somewhere, even if the load current is suddenly reduced by 99.9%.
Many smaller switchers overcome these issues by using relatively small inductors, and operating in what's called "discontinuous mode". The basic idea is that if an inductor is small enough, one can start each cycle with zero current flowing through the inductor, and still put enough energy into the inductor to handle the load's demand for the next cycle. Using a larger inductor would either reduce the amount of power the supply could handle, or else would require it to run at a lower switching frequency.
Other switchers use what's called "continuous mode". These units expect that changes to inductor current will occur slowly, spread out over several cycles. On such units, using a larger inductor may improve performance under stable load conditions, but decrease the unit's ability to respond quickly to changing load conditions.