Does 2 % of duty cycle signal cause the 2 % of power dissipation?
If the device isn't affected by the temperature change then a 2% duty cycle will cause 2% of the steady state power dissipation on average. However for the short time that it is turned on the device will be receiving full power, and this will cause a higher peak temperature rise inside it.
How high internal temperature will go depends on the instantaneous power, the time it is applied for, the thermal mass of the active part (the 'junction' in a MOSFET) and the thermal resistances of whatever 'heat-sinking' it has to ambient (substrate, lead frame, wires, packaging, external heat sink, PCB etc.).
If you know all the thermal parameters then you can calculate the temperature rise by analyzing an electronic 'equivalent circuit' where thermal mass is represented by capacitors and thermal resistance by resistors. Thermal power is 'current', and temperature is 'voltage'. The circuit might look like this:-

simulate this circuit – Schematic created using CircuitLab
40W watts of thermal power (represented by the current generator) is being pumped directly into the junction. Because the junction has low thermal mass (Cj) it will 'charge up' (increase temperature) very rapidly. However it is in close contact with the substrate, which draws heat away (through 'resistor' Rjs) to fill its own thermal 'capacitor'. This continues down the line, with each component initially absorbing heat and then passing it on until finally the heat goes to 'ground' (ambient environment).
At each stage the different capacitances create different thermal time constants, and the resistances cause a temperature rise as the heat flows through them. The junction is small so its temperature rises very quickly - eventually stabilizing when all the heat flows through Rjs to the substrate. But the substrate also soon heats up, raising the junction temperature even higher. The case and heatsink have a similar effect, with each stage causing another 'bump' in junction temperature.
But what does this mean for your duty cycle calculations?
Firstly, you must take into account not just the on/off ratio, but also the pulse time. A duty cycle of 2% with a PWM period of 1 second (20ms 'on' time) might have a quite different effect than the same duty cycle with a period of 1ms (20us 'on' time).
Secondly, you cannot assume that PWM will create a junction temperature proportional to the ratio. The average power might be 2%, but the junction temperature will rise and fall as the power is switched on and off - and you can't tell how much or how fast it is varying by measuring the case temperature.
Bottom line - use the shortest pulse width that still enables you to get accurate measurements, and make the resting time as long as practicable. If you get the same results when the pulse time is increased or resting time reduced (and you are still within the device's safe operating area) then you can use it.