if the input is 25V, it will only be able to output (24.2-24.8)V.
That's not right.
The maximum input voltage is clear enough, the input cannot be higher than that.
The drop-out voltage is the minimum amount the input voltage has to be higher than the desired output. Be aware that the drop-out voltage depends on the current you draw from it.
So if the drop-out is 0.5V at the highest current you will draw and you want 6V out, you need to always at least supply 6.5V.
Then, the maximum voltage differential between in and out may be given for one or two types out there, but it's not likely to happen, because that's what the maximum input voltage and maximum power dissipation are for.
A linear regulator (LDO regulator is a subclass of linear regulators) is not magic, any power drained at the input, but not supplied at the output will be turned into heat. As opposed to a switching regulator, which will - at the cost of extra components and more output ripple in most cases - try to be as efficient about it as possible in the situation.
If you put 20V in and take 2V out, but you want 100mA, that means the LDO has to "absorb" 18V at those 100mA, which makes 1.8W of energy it turns right into heat. So if the regulator is not equipped for that, or the way you mount it doesn't support getting rid of that, it will be destroyed.
However, if you take 2mA out of it, that's only 36mW and that can be handled by any linear regulator, even when it's not mounted at all. Unless, of course, it's a high precision regulator with a maximum of 1mA output, but then it's the limited current that does it, not the power. 36mW is absolutely negligible.