In the circuit you show, the voltage across the load will be about 5mV, with or without the resistor. It won't start to regulate until the load resistor is more than about 135 ohms.
In this circuit with a 1K series resistor to +5, the diode acts a bit like a voltage source with about a 12 ohm resistor in series.. for load resistance that is relatively high.
That dynamic resistance increases with the load current- with a 270 ohm load it will be more like 24 ohms (calculated from 52mV/Idiode- the diode ideality factor of about 2 multiplied by the thermal voltage at room temperature). That means that with a 270 ohm load the ripple rejection is only about -20dB. Load regulation will be similarly poor, but zeners for less than a few volts are horrible in performance or nonexistent so you can't really compare it with a zener. You won't find a 0.6V or a 1V zener.
The diode has a temperature coefficient of about -2mV/°C.
So, it's quite usable in some undemanding applications- for example to determine the current through an LED or to set an overcurrent limit on a power supply, but it won't be mistaken for a precision reference.
This type of circuit (where the zener might be something else) is called a 'shunt regulator' and is mostly used for relatively low currents and certain other special applications. It suffers from a number of disadvantages such as inefficiency if the input voltage is close to the output voltage and the load is variable (because that series resistor has to be very low and the wasted power with a light load is very high).
A popular example of a chip that does this well is the TL431 (or TLV431). Using a couple of resistors you can set an accurate voltage from about 1.25V to some tens of volts depending on the chip. The chip needs 1mA or less to operate.