The shift in energy levels of valence and conductive band does not explain why the amount of light (i.e. the generated photon flux) decreases. From a quantum mechanical perspective, the emitted wavelength is determined by the forward drop (band gap) and as the forward voltage decreases, the emitted wavelength increases.
The photon flux on the other hand is strictly proportional to the electrical current in the diode - each electron has a certain chance to cause the emission of one photon. The "certain chance", commonly called quantum yield, decreases with higher temperatures, which is the effect you are describing.
The major reasons for a decreasing quantum yield are
the dramatically changing lifetimes of radiative and non-radiative recombinations, which leads to non-radiative recombinations taking over (see e.g. Shockley-Read-Hall and Auger recombination)
phonons (crystal oscillation), which can interact with electrons and interrupt a radiative recombination. As those oscillations increase with temperature, fewer electrons participate in light generation
more recent research (2019 - at least after I studied) seems to suggest that transport mechanisms are the dominant contributor to thermal droop (paper I have not read myself)
So, there are two things happening when the temperature increases, which both cause the emitted power to get smaller:
- Fewer electrons participate in light generation and
- The electrons participating in light generation generate photons with less energy (this one does not decrease the LED's efficiency because the consumed power decreases respectively).
It is important to differentiate those two effects, that's one of the key points why we had to come up with quantum mechanics.
I recommend this article and this paper which cover the topic.