I know red LEDs have a higher forward voltage drop than a silicon diode; this leads to about 2.6x more power dissipation at a given current and so 2.6x less current rating, but 20mA is way less than 1A. And yet, a humble 1N4001 won't complain about 1A but a 5mm LED (about the same size) will not last long at all. Why is this?
Packaging: A power diode can have much material (Si and metal) to take the heat where an LED needs a transparent surface and housing to let the light out.
Power dissipation: As you say, Power diodes are trimmed for a low forward (conducting) voltage drop. Si diodes can go as low as maybe 300...500 mV, as opposed to the typical 650...750 mV for small signal diodes like the 1N4148. LEDs have typical forward voltages in the range of 1600 mV...2400 mV. What troubles the chip is the heat generated by the dissipated power: P=V*I. Let's assume (for simplicity) that an LED and a power diode take the same amount of power. With an LED's forward voltage five times greater than that of an "ordinary" diode, the LED can handle only 1/5 of the current.
Process trimming: Also, LEDs are primarily trimmed for a good efficiency (Light output with respect to electrical power input) whereas power diodes are trimmed for maximum power handling capabilities. Power diodes can therefore usually be run at higher temperatures than LEDs. Actually, power diodes start to feel happy at temperatures where LEDs start to disintegrate.
These are just very rough approximations and the folks working on the research of better high-output LEDs probably cringe when they read this, but the examples still show the general answer for your question... And they show what LED research aims at: High-temp chips, innovative packaging techniques to conduct the heat away from the chip, low forward voltages (usually defined by the material and dopants involved in the process -- remember that the material defines the color and that not much stuff is available at all to build some colors), ...