LEDs can be, have been and are used as voltage references. Standard LEDs in various colors are cheap and hence sensible when wanting a low accuracy low voltage reference. Remember that low voltage Zeners have bad knee characteristics and poor tolerances. Zener diodes do make noise when in the breakdown region. In fact, there are noise generators based on this known principle in antique ARRL handbooks. However, is the LED less noisy than the Zener?
(True) Zener diodes are not generally used as noise sources- 'Zeners' for >6V are actually avalanche diodes.
This paper mentions that a 5V Zener has about 15-20dB less noise than a 12V 'Zener'. So, I'll assume that the (dominant in a 12V 'Zener') avalanche noise is not a factor since you are going to use a true Zener diode.
Both an LED and a Zener will have shot noise, which is proportional to the square root of current, so it may well depend on the relative operating currents. There will also be some thermal noise from the resistive component, which would probably favor a Zener over a small LED.
Usually if noise is concern you should follow a relatively low noise source with a conventional low pass filter or something like a capacitance multiplier. That's how reference chips designed for RF circuits achieve uV wideband noise using relatively noisy bandgap references.
I don't think it is as much about noise with a large C than stable bias V.
LED's are better voltage regulators than most small zeners due to size of chip, current rating and lower ESR. Noise could be a factor if large e-cap is insufficient but lower Rs or ESR gives better Q point stability.
ESR is the variable that affects any diode or Zener Vf vs If and is always = 1/Pd which depends on size of chip.
- 5mm IR LED's are 1.1V with about <2~4 Ohms (Zt or Rs or ESR) @10mA and much lower at 100mA
- 5mm Red Yellow around 10 Ohms and 5mm White, Blue around 15 Ohms @20mA
- these tolerances can be 50% for ESR
Whereas small zeners are ~ 20 Ohms in low voltage range.
I can backup with actual part numbers so these are just rule of thumb nominal values.
But since advent of Bandgap reference diodes built into adjustable regulators, this is a better choice.
Quiet depends on many things but mroe important as voltage / current regulator. Diode filter bandwidth depends on ESR*C = T = 3~4f(-3dB)