It depends whether you want static accuracy, or dynamic accuracy.
In order to get static accuracy, you only need very large gain in your feedback loop (in addition to stability of course). This basically means including an integrator in the feedback path, so that at DC, you have the full open loop gain of your feedback amplifier available. After a while, when everything has settled down after a change, you ought to be able to get 4 digits of repeatability at your output terminal with no problems. To turn that repeatability into accuracy, you need a stable reference, and to take care about where you sense the output voltage with respect to voltage drops due to output currents (including the ground sense!)
If you need dynamic accuracy, so that under a line or load transient the deviation does not exceed 1:10000, then good luck, that will be a very interesting problem indeed. You will need to understand loop gain and stability interactions very well. Some topologies are easier to stabilise than others. You will need to define the range of impedances your load can present. The wider range of loads it must work with, the more 'interesting' the design challenge.
A lot of work has been done and is available in the application notes of the big manufacturers (TI, Linear, Analog) on voltage regulators. Read them.