For those unaware, tube amps require the use of transformers (except for certain extreme examples), because the plate voltage is quite high (~300V) and current quite low (~100mA), essentially useless into a low impedance speaker by itself. Tubes are also only "N type" as it were, so push-pull amplifiers can only be done by totem-pole arrangements (awkward to drive), or using a transformer for phase inversion (which, with the transformer being more-or-less mandatory anyway, is fine).
We can refer to these examples, and draw similar conclusions, noting the differences :
https://www.tutorialspoint.com/amplifiers/transformer_coupled_class_a_power_amplifier.htm
https://www.tutorialspoint.com/amplifiers/push_pull_class_a_power_amplifier.htm
Basic operation with a load is, the transformer's magnetizing inductance is charged up to quiescent current, and as the device throttles down, that (inductive) current flows into the load, pushing the plate voltage up. This happens in a controlled manner because the load impedance sets the voltage due to a given current. (Tube amps with local negative feedback, or triodes (which can be considered to have internal NFB), can still control this voltage without a load connected.) Without a load, the voltage is limited only by the impedance of the transformer itself -- which has an LC characteristic, peak voltage given by Iq * sqrt(L/C). Which isn't necessarily going to be destructive or anything, but, as is usually the case -- it depends, and it could lead to arc-over, typically at the tube socket or something like that. Rarely internally, causing device damage. (For the transistor case, avalanche may occur, more likely to cause damage.)
In short, particularly when overdriven, the circuit resembles a boost converter with no load, and the flyback pulse can reach dangerous voltages.
For the push-pull case, note that one side turns on while the other turns off. With normally no quiescent current in the transformer (the currents in the two sides are balanced for zero net magnetization, and class B operation is possible so that peak signal current can be many times Iq), flyback isn't an issue, and for the transistor case at least, the opposite side acts to clamp that voltage.
We encounter a difference with tubes here: vacuum tubes are inherently [series] diodes (no current flow in reverse) -- the inverse of MOSFETs (inherent parallel diode, full current flow in reverse). (BJTs can handle some reverse current, but only when the base is forward-biased. They're kind of inbetween I guess you could say.) So we can have the situation that one side turns off and the other turns on, but the voltage dips too low, reverse-biasing the "on" tube. This doesn't result in dangerous voltages, so much, but with the high grid voltage (it's "on"), cathode current is demanded -- but there's no plate voltage to absorb it, and consequently a massive current flows into the screen grid instead (for tetrodes+; N/A for triodes, which are fine here). This can result in "toaster grid" and subsequent destruction.
There could also be instability where, due to the light load, feedback goes unstable and oscillation results; and with help of the above mechanics, a relatively large voltage could develop (i.e. more than twice B+ voltage). Which might go with toaster-grid operation too, so, all around not a good time. This does require that the circuit is poorly compensated, which as usual, depends.
