The part of the chip that does the management (the comparator, the voltage reference, the PWM generator and various low-level interface circuits) take some amount of current i.e. they are not self-powering and that's basically the main clue. They require current and that current/power quite often is provided by a low-level LINEAR regulator.
Given that the efficiency of that (or any) linear regulator worsens proportional to input voltage, then that linearly regulated power eats into the overall efficiency of the full converter and, it's more obvious at higher input voltages and lower output load powers.
Here's what the quiescent power required by the converter is at various input voltages: -
So, when Vin = 19 volts, the device is "wasting" 2.2 mW and, given that in your 3.3 volt efficiency diagram (on the right) there is a load power of 33 mW at 10 mA load you can begin to see that the "management stuff" is becoming a significant factor. The actual implied losses when producing 10 mA at 3.3 volts will be from the MOSFETs and inductor. They don't come for free either.
So, if I were to add things up, to make an output power of 33 mW at an efficiency of 84% requires a total power in of 39.3 mW. Given that the management stuff consumes about 2.2 mW that leaves about 4 mW wasted in the MOSFETs and inductor.
On a high load power, the 2.2 mW consumed by the management stuff becomes trivial because most of the losses are being created by the MOSFETs and inductor.
For instance, with a 4 amp load and Vout = 3.3 volts, the output power is 13.2 watts. The graph's stated efficiency is 94% hence, the total power is 14.04 watts. In other words 0.84 watts are wasted by MOSFETs and inductor and, it's just not worth trying to factor in the 2.2 mW of the management stuff. But clearly they are significant when you are talking low output powers.