Knowing that I only have op-amps, comparators, resistors, capacitors, and simple diodes at my disposal, is there a simple way to amplify the small oscillation of such a signal without amplifying the mean of the oscillation? (I would like the curve that the oscillation follows to remain the same)
It sounds like you are looking for a high pass filter, but more crucially, one that has a DC gain of unity to retain the input DC offset at the output.
A standard non inverting amplifier with an AC capacitor can achieve this with minimal parts. Resistor values can be tweaked to achieve the desired gain of the oscillation (passband signal) using the standard non inverting opamp equations. Gain at DC is always unity. The optimum capacitor value will depend on how quickly you want the circuit to be able to respond to the DC input (but ideally transitioned to acting as a short circuit by the oscillation frequency).