You need to look at the silicon stack up for P-channel and N-channel JFETs.
You can see the PN junction formed between gate and channel. That's a diode and needs to remain reverse biased otherwise it will be a shorting diode and fry. Since that PN junction is flipped between P-channel and N-channel JFETs that means the voltage applied between gate and source also needs to be flipped to remain reverse biased.
Also notice that the JFETs is symmetrical so source and drain are interchangeable. So how do you consistently know which end is supposed to be the source?
By convention the source is labelled to be whichever the reference terminal for the gate, just like a MOSFET. Unlike a MOSFET it's not immediately evident which end it is because the device is symmetrical and would work either way. So it's not specific to the construction of the device. Instead, it's specific to the way the device is used in the circuit. You know which end is the reference terminal for the gate because it's the end that forms a reverse biased PN junction with the center slab of silicon (which is the gate, obviously).
This in turn causes the relative voltage between the drain-source assignments to be flipped between P-channel and N-channel JFET.