Does anyone know how to calculate the clamp voltage? I read some
papers and documents. Some of them use \$n×Vo\$ to calculate the
\$n×Vo\$ tells you the secondary-to-primary reflected voltage in normal operation and, for the ideal case of zero leakage inductance. The clamp voltage cannot be lower than this.
Using your numbers, if Vs is 330 volts and Vo is 24 volts, when the transformer flies-back, the voltage seen at the drain of the main MOSFET is 330 + (24 × n).
You have specified Np:Ns as being a 5:1 step-down and therefore, the output voltage of 24 volts is stepped up by 5 (to produce a peak of 120 volts) and, this is superimposed on the 330 volts at the drain. Hence, the peak voltage at the MOSFET drain is 450 volts. You can't clamp this; it is a fact of life; a fundamental aspect of fly-back converters.
So, don't clamp this voltage because, this is how a fly-back circuit works.
What you do need to clamp are voltages in excess of 450 volts because, these are formed by the primary leakage inductance (adding to the fly-back voltage) and, may be detrimental to your MOSFET. Ideal scenario (no leakage): -
Image from my basic website.
So, your clamp voltage needs to be higher than 450 volts (with respect to ground) but lower than the MOSFET maximum DS voltage.