1) Why would people want to use polarized capacitors?
Because, usually, they have more capacitance per volume and are cheaper. Search for values like 1000uF, 2200uF, 4700uF or even bigger of ceramic ones and will find the answer. If your value is low, you don't need polarized caps.
2) What is the purpose of the transformer?
If the purpose of circuit is only a voltage doubler, its for isolation. But you may achieve bigger voltages ratio with the transformer. An isolation transformer is a safety device. Since the right part of circuit is floating, if you touch only at one point of the circuit you will not be electrocuted. In spite off, for example, if you touch the positive side of the last capacitor it will be grounded and the negative will be -V.
3) What happens if I reverse the orientation of both diodes? It seems that it inverts the sign of the output voltage, but this is only possible using non-polarized capacitors, which brings me back to my first question.
If you reverse the diodes you must reverse also the capacitors.
Also, generally, non polarized capacitors have lower inductance and resistances. So they act faster and are better suited for high frequencies circuits.