You can try something like this:
You need to make sure that the pass transistor (M1) can handle the load current. M2 and M3 are low profile FETs (no need for high current capability) used to disable M1 when the 24V source is connected. This will automatically disconnect the 28V source when 24V is present.
Here are the situations you can have (28V is always present as stated in your post):
(1) 24V source is not connected: In this case, M3(NMOS) is OFF because R5 pulls it to ground. R4 keeps M2's(PMOS) source and gate at the same potential, and therefore, stays OFF. The only transistor that is active is M1(PMOS) because R3 and R1 create enough voltage difference between the source and the gate (~10V).
(2) 24V source is connected: In this case, R6 and R5 turn on M3, which as a result turns on M2. R4 and R7 ensure you don't over-voltage M2's VGS. Turning on M2 means that the source and the gate of M1 are at the same potential, and therefore stays OFF, disconnecting the 28V source. D1 provides a path for the 24V source to the output.
The resistor voltage dividers keep the voltages below the max ratings of the transistors. D1 blocks the VOUT node from messing with the 24V node, more specifically, keeps it from creating oscillations when the 24V is not present.