If the lowest frequency of the sinewave is audio for example, all you need is a decoupling capacitor and a resistor that connects to the 2.5 volt DC offset. Here's how to remove a dc level: -
The output DC level can be restored by attaching the resistor (R) to 2.5 volts instead of 0 volts. If you need to connect to a load that might alter the planned DC offset then maybe you will need to add an op-amp buffer.
A 10 uF capacitor and 10 kohm resistor will have a 3 dB point at 1.59 Hz and this would be suitable for audio signals: -
Can I somehow use the LM324N by any chance since I have that one
I don't see any reason to use an op-amp unless you want the input frequency to go down to 0 Hz or you have a significant load to be attached to the output.