C3 turns the circuit into an oscillator. To understand this you need to think hard about the role of the emitter and the thing you have to think hard about is this - the emitter (as well as the base) is an input to the transistor that is amplified by the transistor.
An ac voltage (superimposed on a dc voltage) between base and emitter is amplified by transistor action - people usually talk about an ac voltage applied to the base being amplified by the transistor but, in reality the base current is due to a voltage difference of about 0.7 volts between base and emitter. If the signal on the base (or emitter) alters the base current then the transistor will amplify it it.
If you held the emitter voltage constant and raised the base voltage there would be an increase in base current that would cause a greater collector current. If you held the base voltage constant and increased the emitter voltage there would be a decrease in collector current.
So when base voltage increases, collector current increases and there is a tendency for collector voltage to drop - i.e. the voltage on the collector is inverted compared to the base BUT it's not inverted when you raise the emitter voltage - collector voltage (signals) are non-inverted (and amplified) versions of the signal on the emitter - this means that C3 creates positive feedback and, positive feedback (in this type of circuit) creates an oscillator.
The modulation is a little subtler - there is always a few picofarads capacitance between collector and base and this capacitance gets smaller as the collector-base junction gets more reverse biased. This means you can tune the oscillator a bit by raising or lowering the base voltage - the microphone does this with the audio signal it produces and the CB capacitance is modulated at the same rate as the signal from the microphone. This undulating CB capacitance is, effectively in parallel with C2 and bingo, you get FM (plus a little bit of AM too.