I remember a label printer I dissected as a kid having the same. From the position of the LEDs, it looked like they were pretty close to the EEPROMs on the device. My best guess hence is: This might actually be a way for the manufacturer to, at low cost, add a programming header to the devices that allows for different functionalities being programmed in a manual labor assembly line, where you just want to decide at last minute, before you put the back cover on, into which market you'll sell that product.
Why solar calculators? If they ran QA on those, they'd want a way to test them, so one would assume you just want to put your calculator in a light box, and run a few tests. Now "run a few tests" is a bit complicated with a device that runs at wildly varying voltages, and probably also wildly varying clock rates: you either end up somehow electrically coupling the device with sensitive connectors (which are expensive) to something that does the level shifting in your test stand, or you do the level shifting in your calculator and end up having an unnecessary level shifter in every calculator. Or, you just add some wireless link, like, in the simplest case, two LEDs, one used as a photodiode (receiver), one as a Light emitting diode (transmitter). You place the half-assembled calculator manually on a frame that is being lit from a below, and communicate with the LEDs in the shadow of the calculator itself.
My other best guess is: LEDs have become a mass product, and they come with a relatively well-defined band gap. So if you need a 2.15 V voltage reference for whatever, why not go and buy the cheapest yellow LED you can find?