Piezoelectric effect works both ways:
Applying voltage across a piezo material will cause it to change shape (usually expand or bend).
Bending or squishing a piezo material will cause it to generate charge and current.
In the case of a crystal oscillator, the crystal is cut in such a way to resonate at the desired frequency.
If you want to think about it in the time-domain, if you apply a voltage pulse to the crystal, it will bend and then resonate, convert these vibrations back into voltage, and it will output a damped oscillation at its characteristic frequency. It is a bit like a tuning fork: you hit it, then it goes "dunnnnnnnnnn" as it keeps oscillating. Without any extra energy input, the the energy will dissipate, and the oscillations will dampen and eventually stop.
In the frequency domain, this translates into an impedance curve with a high-Q peak and a sharp phase shift at the resonance frequency.
The role of the feedback amplifier is to feed back the signal into the crystal so the oscillations keep going. It's like pushing someone on a swing: to keep it going, the push has to be applied with the correct phase.
As per Nyquist's stability criteria, if a feedback circuit has 180° phase shift and unity gain at a frequency, then it will oscillate at this frequency. The sharp impedance peak and sharp phase transition from the crystal at resonance create this condition, so the amplifier feeds back the signal into the crystal with the correct phase to sustain oscillations.
What you say about noise is more about how the circuit starts. When it is powered up, the crystal is not oscillating, but the circuit will amplify its own noise until there is enough to excite the resonance.
