I'm using an ADM3053 isolated CAN transceiver in my design, which, due to layout constraints, is located ~6" away from the board CAN connector. The CAN_H and CAN_L traces are routed as a 100Ohm differential pair (100Ohm was the most I was able to achieve with my layer configuration) on the bottom layer of my 6-layer board (SIG, GND, SIG, PWR, GND, SIG), with a closely coupled GND plane as reference (layer 5).
As the CAN signals are isolated from the rest of the board, the CAN_H and CAN_L traces need an isolated GND reference (CAN_GND), as opposed to the system GND on layer 5. Am I better off running a thick trace underneath the CAN_H and CAN_L pair on layer 5 all the way from the ADM3053 transceiver to the CAN_GND pin on the CAN connector, or running CAN_GND as guard/return traces alongside the differential pair on layer 6?
My concern with the former is the disruption to the otherwise uniform GND plane on layer 5. The area of the board between the ADM3053 and the CAN connector is sparsely populated on layer 6, so I think the CAN_GND trace on layer 5 could be routed in such a way so as to minimise the impact of the disruption to the GND plane.