I am designing a Ethernet PCB with external magnetics on a 4 layer board. On the PHY side I have the system ground on the adjacent layer under the two differential pair for the return path. How about in between the connector and the magnetics? Should I pour a chassis ground plane under the traces or am I not suppose to have any copper?
I suppose the first point that comes to mind: it may be difficult to achieve isolation requirements, pouring ground under the media traces [on nearest mid layer]. That said, you could still use the "Bob Smith" common node (to which the unused pairs, and center-taps, are terminated, with the 1nF to ESD ground) as a ground plane for these.
You generally won't want to route media traces as diff pair without ground, as the impedance will be too high. Edge-coupled traces with ground, couple poorly to each other; even without ground, they still couple poorly, requiring large trace widths for impedances like this. You can however route them top-mid or top-bottom as broadside-coupled pairs, which will see a more modest trace width.
Finally, if there's very little distance between connector and transformers -- mismatch really doesn't matter. And since conventional 100/gig Ethernet only uses around 100MHz bandwidth, "little" is perhaps 10cm or less. Ethernet may be high bandwidth in relative terms, but it's nowhere near as intensive or demanding as a DDR3+ or PCIe interface is!
There is also an argument to prefer the no-ground case in terms of common mode impedance: we're probably not talking much coupling distance here (i.e. maybe some ~cm of pair over plane, wired to whatever ground return you choose), so again it doesn't matter much, but the case could be made that no-ground is preferable as it's consistent with the environment of the cable itself. That is, you can avoid introducing more and longer impedance discontinuities. The connector and transformer are already two, and coupling the pair to a ground adds a third. Discontinuities mean more impedance changes vs. frequency, and maybe that worsens CMRR at some frequencies.
Or conversely if shielded cable and connector are in use, extending that shield onto the board (and tying the "Bob Smith" node to it) would be equally logical.
To be clear, chassis ground is most likely the "ESD ground" mentioned above, i.e. where the "Bob Smith" common node should be bypassed to. Because of isolation requirements, chassis ground probably shouldn't be used for this directly, but due to the capacitor, these two nodes will have similar RF voltages.