I just built a small 5.1V regulator with LM317, using 220 and 680 ohm resistors. However, I measure 8.6V at the output without a load connected. Is this normal, i.e. under a load (about 120 ohm) it would drop to 5.1V? The input is a 9V/300mA unregulated wall wart, which reads about 13V without a load.
You need a minimum load of 10mA for it to be stable. A tip is to add a LED and run it on 5mA. You'll know whether the supply is on, and it will add additional load to make it stable.
Note that the adjustment resistors of 220+680 ohm also give some load, of about 5.6mA. Together with a LED you get more than the required 10mA.
The regulator works mostly like a resistor which is automatically adjusting itself so if there is too little current the voltage drop on the regulator will be too small to regulate the output.
It is possible to make regulators that work without load but they tend to suffer from high quiescent current.