The capacitors on the input provide a near reservoir that doesn't have a lot of inductance or resistance in series with it. This is important when the load suddenly changes the current it is drawing. With enough inductance in series with the input, a sudden increase in load current can cause a sharp negative glitch on the input voltage, which in turn can make the regulator not work. A cap physically near the input can provide the sudden burst of current while the main feed has time to catch up. Without the right input capacitance, the regulator can oscillate and otherwise not operate to spec.
The capacitors on the output smooth the output voltage and are part of the closed loop control of that voltage. There usually needs to be some minimum capacitance just to guarantee stability of the control loop, but more will do a better job of keeping the voltage steady despite high frquency variations in the load current. The active electronics in the regulator only works up to some frequency. Beyond that, the output caps keep the voltage steady.
The reason for two different caps in each place are to get both a reasonable amount of bulk storage and good high frequency response. This is particularly important on the input, especially when there is some distance between there and where the input voltage is ultimately coming from. The electrolytic provides the bulk storage, and the ceramic provides low impedance at higher frequencies. For most such regulators, there is little point in adding a electrolytic across the output. Usually a few µF of ceramic are good enough to keep the regulator stable, and the low ESR of the ceramics provides the high frequency response that the regulator can't actively handle.