Theoretically it should be possible to achieve high efficiency with the right parts. I reproduced your design in LTspice using 'real' components. The calculated efficiency was 81.2%. Here's the circuit:-
I chose an STP11NM80 simply because it was the highest voltage NMOSFET in LTspice's database. However this FET has much lower RDSON and total Gate charge than the IRFPG50. The importance of this becomes apparent when you look at the Drain current waveform:-
When the FET is switched on it passes a current equal to the (average) load current, which in this case is 5 A. With the STP11NM80's RDSON of less than 0.4 Ω the conduction losses aren't bad. The IRFPG50's RDSON can be as high as 2.0 Ω so its conduction loss will be much higher. However with a duty cycle of just under 2% the average conduction loss is still very low (the diode conducts the output current for the other 98% of the time and the inductor 100% of the time, so their conduction losses are far more important).
More concerning is the huge current spike you see at switch-on, mostly caused by the flyback diode's reverse recovery as the voltage across it goes from ~+1.5 V to -600 V. The 38 A peak is just within the STP11NM80's peak current rating, but well over the 25 A rating of the IRFPG50.
The individual component power losses in my simulation were:-
- FET: 1.8 W
- diode: 7.2 W
- inductor: 2.7 W
- capacitor: < 0.01 W
So in this case we see that the diode is the major loss contributor, followed by the inductor and then the FET. With different component choices the rankings could change and total loss could be higher or lower.
So much for the simulation. In the real world you have more problems. You need to control the FET somehow, which requires another power supply just for the controller - which has to boost the Gate voltage above 600 V to drive the NMOSFET. If for any reason voltage regulation fails the output could rise to 600 V with no current limit.
In a practical circuit the FET's switching speed will be limited by the driver, and perhaps also to reduce EMI. This will increase FET power loss during switching when it has both high voltage across it and high current going though it.
These problems can be solved solved by using a transformer, which improves efficiency by running at a higher duty cycle, can have an auxiliary low voltage winding to power the controller, and provides galvanic isolation. You might prefer an inductor to save space, but a 500 mH inductor rated for 2 A is not small either.