(100*10%)^2 * 4.5mohm P = 0.45W is wrong because the square of a mean value is not equal to the mean of the squares.
You are interested in the mean power that's why you first have to calculate the power for continuous current and then multiply by 0.1 for the duty-cycle, i.e. you have to calculate
\$(100A)^2 \cdot 4.5m\Omega \cdot 0.1 = 4.5W\$
in order to get the mean power dissipation in the MOSFET.
EDIT:
For calculating the temperature rise you must not use the
derating factor (units [
\$W/°C\$]; it just gives you an estimate how much the max. power dissipation has to be derated per each °C above 25°C)
like you have done, but the
thermal resistance \$R_\theta\$ (units [
\$°C/W\$]; it tells you how much the temperature rises per Watt).
If you don't use a eat sink the datasheet tells you that thermal resistance from junction to ambience is
\$R_{\theta JA} = 62°C/W\$
If you use a heat sink the datasheet tells you that thermal resistance from junction to heat sink is
\$R_{\theta JS} = R_{\theta JC} + R_{\theta CS} = 0.402°C/W + 0.5°C/W \approx 1 °C/W\$
So without heat sink junction temperature rise with respect to ambience would be
\$4.5W \cdot 62°C/W = 279°C\$ (which is much too high; i.e. you need a heat sink).
With heat sink junction temperature rise with respect to heat sink would be
\$4.5W \cdot 1°C/W=4.5°C\$.
Of course it depends on the thermal resistance of the heat sink what you will get as total temperature rise with respect to ambience.
Still you have to take care that instantaneous power (=45W) is within limits during the on-time.