At a high level, the main difference is that output from the collector can give you voltage gain, while output from the emitter gives you current gain, but not voltage gain.
Sometimes it's useful to have the same signal but at a lower impedance. In that case a emitter follower (output from emitter) can be useful. In the end, most "amplifiers" need overall voltage gain, so output from the collector is going to be needed somewhere in there.
Note that in the second stage of your circuit, the collector resistor is 10x the emitter resistor. Since roughly the same current flows thru the collector as emitter, the voltage variations as the collector will be about 10x higher (voltage gain of 10) at the collector than the emitter.
Your circuit actually has even higher gain, due to additional resistor and capacitor in series on the emitter. Since the capacitor blocks DC, only the 1 kΩ resistor matters in setting the DC operating point of the transistor, also called biasing the transistor. The 470 Ω resistor with 10 µF cap in series effectively make the emitter resistor lower at audio frequencies. This makes biasing easier and more predictable, but still gives higher gain at the audio frequencies where we care about gain.
The rolloff frequency of 470 Ω and 10 µF is 34 Hz, so this isn't quite a "HiFi" audio amp. At low frequencies, the gain of the second transistor stage is about 10. Starting at around 30 Hz, the gain goes up so that eventually the emitter resistor looks like (1 kΩ // 470 Ω) = 320 Ω, for a gain of about 30.