as the title said, I'm facing this problem. I'm controlling a Peltier element (91.5W at 15V, what should give us aprox. 4A at 12V) with two different MOSFETs: IRF3205 (Rds(on) = 8 mOhm) and FQP30N06 (20mOhm). Using the formula Tjunction = Rja * (Rds_on * 4A²) + Tambient, the second MOSFET should raise its temperature to 45°C at 4A, but in real life, it's getting very, very warm. As my multimeter isn't reliable for measuring current, I decided to put a 0.235R resistor between Source and Vss for measuring current by the voltage drop across the resistor, and got 0.8V, which means aprox. 3.4A, and guess what? The MOSFET doesn't heat up as much as before: with the Current Sensing Resistor, the heat dissipated is what it should be by the formula. My question is: why the MOSFET gets so hot without the 0.235R resistor? Before you ask, the MOSFET is full on (gate connected to 12V by a 3k9 resistor. No PWM or something like that. It's always on). And yes, the IRF3205 gets hot too.
You are assuming that the current is constant. In fact if your supply has ripple, the dissipation will be much higher than if the voltage is steady, because the power is the integral over time of the currents squared times the Rds(on) resistance (which might be 45m\$\Omega\$ as others have pointed out). If you have a true-RMS meter you can measure the RMS current and voltage (current is more important than voltage across the device, but voltage through a sense resistor is fine) and determine the heating effect.
It will also change with the temperature of the Peltier so you may not have done an exact A-B comparison.
Even at 45m ohms, you should probably have a small heat sink on the MOSFET or get a better MOSFET, if the current is in the 4A RMS region.
By the way, ripple which causes the MOSFET to heat will also seriously negatively impact the efficiency (miserable at the best of times) of the Peltier device. It should certainly be kept to less than 10%.
Your FQP30N06 does not have an Rds ON of 20 mOhms. With your Vgs = 12Vdc, I would expect an Rds ON of closer to 30 mOhm - not a huge difference but there is a caveat.. that graph says Note: Tj = 25 degC. You calculated your Tj at around 45 degC, making that value of Rds ON invalid. In reality, as Tj climbs (espeically to 45 degC) I would expect Rds ON to increase. This is a form of a positive feedback loop, as Tj increases, Rj increases, which causes Tj to increase etc.
Also, assuming the Rds ON of your FETs is fairly low, adding such a big "current sense resistor" will really limit the current (series current is now R_peltier + Rds ON FETS + 235 mOhms). This means lower current, causing a lower Tj and therefore a lower Rds ON. Even a 500 mA difference in current can make a big impact.