I've read that when electrons enter from metal to semiconductor type N, they gain thermal energy and make that side cooler and the reverse happens when they leave semiconductor to the conductor. Why does this happen? How do electrons gain, hold, transfer and lose this thermal energy in this process?
The valence band is shown in the shaded area. In conductors the conductance band and the valence band overlap. In semiconductors there is an energy gap between the conductance and valance bands, as shown in the figure. A conducting electron must gain energy as it transitions from the conductance band in the conductor to the conductance band in the semiconductor and must lose energy vice versa. This energy is the thermal energy, hence heating or cooling. A similar argument applies for p-type semiconductors.
This appears to simply be an observable effect. The best explanations are probably in mathematical equations, but the simplest phrasing I could find was that:
"The Peltier effect is caused by the fact that an electric current is accompanied by a heat current in a homogeneous conductor even at constant temperature. The magnitude of this heat current is given by \$\Pi \cdot I\$"
(Pi times current), where \$\Pi\$ is the peltier/seebeck coefficient of that material.
This coefficient can be positive or negative and as a result materials can be paired and alternated in a zig zag to produce a favorable series electrical arrangement along with a parallel thermal arrangement.
I've been unable to find anything regarding the precise mechanism (it makes sense to me that electrons would carry thermal energy in the direction they are flowing, but I'd like to know why thermal energy would flow against the actual flow of material). It could be an effect at the barrier regions of the materials as some of the other papers seem to suggest, with the electrons accepting a transfer of thermal energy in order to cross the barrier between different materials. This would appear to better account for negative coefficients. I think you should consider migrating this question to the physics stack exchange as they may be able to provide a more in depth model.
Better question is, how do electrons carry thermal energy in any metal?
On the other hand, we could crudely answer OP's question in terms of the semiconductor physics behind LEDs and PV cells. In LEDs, if a wandering electron falls into a hole, both carriers cancel, and this releases energy, either EM waves (photons) or lattice vibrations (phonons.) The opposite is also true, where either lattice vibrations or EM waves can be absorbed, elevating a carrier to a higher-energy band and producing an electron-hole pair.
The hot-junction of a Peltier device is like an "LED for phonons," emitting energy as the carriers cross a junction and fall down an energy-hill (the built-in junction-potential,) falling to a lower energy-band. The cold junction is opposite, where carriers can cross the energy-hill but only if they randomly absorb ambient energy and rise to a higher energy-band. Don't forget that Peltier arrays involve Nonrectifying Junctions, where no depletion zone is blocking reverse currents.
Won't ordinary diodes exhibit these effects? Yep. But any thermal emission at the PN junction is exactly countered by thermal absorption at the metal-N and P-metal connections, where the diode's metal leads connect to semiconductor. If you made yourself an inch-long germanium diode with a pn junction in the center, you should notice that when the junction becomes warm, the metal contacts both become cold. And no diode is needed: your junction could have been (++p)-metal-(--n).