When you switch the circuit on, currents start to flow. Together with the voltages across the components, this causes them to dissipate power which causes them to warm up.
That is perfectly normal and to be expected.
Read the manual of an accurate voltmeter or reference voltage generator and it will state that you must leave the device on to warm up. Often that needs to be for at least one hour before the specified accuracy is guaranteed.
The resistors in your circuit will not be very sensitive to temperature changes but the transistors will be. The forward voltage of a diode and therefore also the Vbe of a transistor will have a typical temperature coefficient of -2 mV/K meaning Vbe decreases 2 mV for every degree Kelvin (or Celcius) temperature increase.
So it is not the opamp itself that's causing this behavior. In your (Bandgap reference) circuit, the opamp will do "whatever it takes" to make the voltages at its inputs the same. As the transistors change their behavior, the opamp needs to adjust its output voltage.