The SERDES/transceiver design inside the Ethernet MAC controller

Question

I'm a little confused about the "SERDES" interface between MAC and PHY chip, and I drew some figures to illustrate the connections which confuse me as shown below. The MAC controllers in 3 figures are the same, but the transceivers inside the MAC are not so straight forward to me. The 1st and 2nd figures are normal application which transmits the data through copper media with coded information (through PCS/PMD/PMA inside the PHY chip). The interface between the MAC and PHY is SGMII or XAUI for 1G and 10G base-T Ethernet. However, the 3rd figure confuses me. There are applications where the MAC is connected to the optical-electrical conversion element, and transmit the data with lasers and fiber cable.

Questions:

1. My first question here is, where is the PHY function now (PCS/PMD/PMA) in this situation? Looks like the data is transmitting directly from the "SERDES" of MAC to the other MAC's SERDES, only the media transformed from electrical to optical, without any coding or manipulation as PHY chip does on copper application. Is there any reason why we can do that (or I should say, why we need PHY chip to transmit the data through copper media)? Is it because the 1000 Base-X follows the physical layer of FC0 (FC-PI, fibre channel physical interface) and not the OSI model? If that’s the case, why we don’t need the PCS/PMA/PMD stuff for fibre channel application? And any special design in the transceivers to achieve such goal?

2. The second question is, what's the difference between SGMII/XAUI and SERDES? It seems that the MAC chip can support both function on the same high-speed lanes. The SGMII/XAUI are usually used for the connection between MAC and PHY chip, where the “SERDES” is used for MAC direct connection. I used to believe that interface like SGMII or XAUI are all SERDES, but it confuses me now.

3. The other question is, there's another connection called DAC (Direct Attached Copper), does it have anything to do with MAC to MAC connection? The signals now connect the MAC without PHY, why can we do that? If we can do so, why bother to add external PHY on the RJ45 scenario? Is is about the transmitting distance?
4. Do we still need DC balance in fiber signaling? If not, why bother to encode the data to 8b/10b or 66b/64b? So many questions listed since I’m still new to the networking area. Hope someone can help me with this.

======================================================================= [Extented content]

@alex.forencich

So, the reason why we can use the MAC chip to directly connect to the SFP module(without PHY on the module, only optical transceivers) is that the MAC chip is supposed to have a built in PHY, is that correct?

In other words, can I assume that the data encoding, formatting, etc (all the PCS and PMA functions) are all done in the MAC chip now, instead of doing that in the external PHY chip?

If so, looks like 1000basex do need to go through PCS and PMA before the data can be sent to SFP module, is that true?

The key point which confuses me earlier is that I used to think that 1000base X didn’t require PCS and PMA, and can be connected directly to the SFP module to transfer the data from MAC logic. I grep a figure from Intel and Synopsys website and which may explain that the 1000Base-X still need PCS/PMA functions before going out the chip. If my assumption is correct, the common MAC chip may have built in two kinds of PHYs, and have a switch inside the chip to select which interface it should be running on by detecting the outside modules. For example, if it detects the SFP module with optical devices only, it’ll be configured to be a XFI interface. Please correct me if anything wrong here.

And, the reason why we need external PHY for RJ45+copper cable is because we need to transform SGMII into MDI interface to meet the requirement of the 1000Base-T requirement, right? In this case, the internal PHY may be configured as a pass-through mode, and use SGMII from MAC module to external PHY chip. Not sure if that’s correct.

Answer 1

My first question here is, where is the PHY function now (PCS/PMD/PMA) in this situation? Looks like the data is transmitting directly from the "SERDES" of MAC to the other MAC's SERDES, only the media transformed from electrical to optical, without any coding or manipulation as PHY chip does on copper application. Is there any reason why we can do that (or I should say, why we need PHY chip to transmit the data through copper media)? Is it because the 1000 Base-X follows the physical layer of FC0 (FC-PI, fibre channel physical interface) and not the OSI model? If that’s the case, why we don’t need the PCS/PMA/PMD stuff for fibre channel application? And any special design in the transceivers to achieve such goal?

SGMII and 1000BASE-X are almost identical. The only real difference is autonegotiation. Outside of that, the wire format is identical. In this case, it might be better to consider that the MAC has a built-in PHY that provides the SGMII interface, then another PHY converts SGMII to some other format. If you need 1000BASE-X, then you can configure the PHY in the MAC to speak 1000BASE-X instead of SGMII, and you're good to go.

I will note that there is some blurring of the lines here. It is very common for the MAC and PHY to be integrated on the same chip, and it is also becoming more common to use PHY attach connections that are similar to the wire protocol (SGMII, XFI). This is especially true for higher line rates such as 100G Ethernet. In this case, the PHY chip mainly serves to regenerate the signal for signal integrity purposes as well as possibly providing autonegotiation functionality.

The second question is, what's the difference between SGMII/XAUI and SERDES? It seems that the MAC chip can support both function on the same high-speed lanes. The SGMII/XAUI are usually used for the connection between MAC and PHY chip, where the “SERDES” is used for MAC direct connection. I used to believe that interface like SGMII or XAUI are all SERDES, but it confuses me now.

SGMII is a single SERDES lane of 8b/10b encoded data at 1.25 Gbps. XAUI is four parallel SERDES lanes of 8b/10b encoded data at 3.125 Gbps. SGMII is for gigabit, XAUI is for 10G. SERDES is just a generic term for serializer/deserializer. Again, consider the direct connection case as the MAC having a built-in PHY.

The other question is, there's another connection called DAC (Direct Attached Copper), does it have anything to do with MAC to MAC connection? The signals now connect the MAC without PHY, why can we do that? If we can do so, why bother to add external PHY on the RJ45 scenario? Is is about the transmitting distance?

A DAC cable is a replacement for a fiber optic cable and transceivers. This connects things exactly like inserting a pair of optical modules connected by a fiber. This is considered to be a PHY to PHY connection. The external PHY is used when you need to use an RJ 45 cable for whatever reason. For distance, you would use optical fiber.

Do we still need DC balance in fiber signaling? If not, why bother to encode the data to 8b/10b or 66b/64b?

Line codes do a lot more than simply provide DC balance. They ensure a minimum transition density, which is imperative for clock data recovery at the receiver. They also provide framing, a way to recover bytes from the raw stream of bits, as well as control characters to delimit protocol frames and encode status and error conditions.

To your edits:

So, the reason why we can use the MAC chip to directly connect to the SFP module(without PHY on the module, only optical transceivers) is that the MAC chip is supposed to have a built in PHY, is that correct?

That's essentially correct.

In other words, can I assume that the data encoding, formatting, etc (all the PCS and PMA functions) are all done in the MAC chip now, instead of doing that in the external PHY chip?

I mean, this is where the lines between the layers are a bit fuzzy. The MAC and PHY do not have to be separate chips. "MAC" and "PHY" refer to sets of functionalities, there is no requirement that they exist on physically separate chips. It's certainly possible to put the MAC and the PHY on the same chip. Most NICs these days are single-chip solutions - you have the whole NIC host interface and DMA engine, MAC, and PHY all on the same piece of silicon. For example, one chip with 16 lanes of PCIe on one side and four 25 Gbps serdes on the other side, directly connected to a QSFP28.

If so, looks like 1000basex do need to go through PCS and PMA before the data can be sent to SFP module, is that true?

They do. But these layers could be rolled in to the same physical chip as the MAC. In fact, you could even make a MAC that is designed to directly connect to a serdes without any GMII/XGMII/etc. interface, even on-chip. Like this one, for example: https://github.com/alexforencich/verilog-ethernet/blob/master/rtl/eth_mac_phy_10g.v . AXI stream interface to core logic on one side, raw serdes interface for 10GBASE-R on the other side, with no extra stuff (XGMII) in between.

The key point which confuses me earlier is that I used to think that 1000base X didn’t require PCS and PMA, and can be connected directly to the SFP module to transfer the data from MAC logic. I grep a figure from Intel and Synopsys website and which may explain that the 1000Base-X still need PCS/PMA functions before going out the chip. If my assumption is correct, the common MAC chip may have built in two kinds of PHYs, and have a switch inside the chip to select which interface it should be running on by detecting the outside modules. For example, if it detects the SFP module with optical devices only, it’ll be configured to be a XFI interface. Please correct me if anything wrong here.

It does require the PCS and PMA. These are logical layers, they can be mixed together in an actual implementation. And it is certainly possible to reconfigure the interface depending on what's connected, even to totally switch the "role" between a PHY (1000BASE-X) and a PHY-attach (SGMII).

And, the reason why we need external PHY for RJ45+copper cable is because we need to transform SGMII into MDI interface to meet the requirement of the 1000Base-T requirement, right? In this case, the internal PHY may be configured as a pass-through mode, and use SGMII from MAC module to external PHY chip. Not sure if that’s correct.

Pretty much. 1000BASE-T is far more complex electrically than a pair of 1.25 Gbps serdes - it requires a lot more pins, analog components including directional couplers, DSP for echo cancellation, etc. 10GBASE-T even more so. So it can make a lot of sense to integrate one or more SGMII PHYs on a chip, then use separate off-the-shelf PHY chips to convert to BASE-T than to try to integrate all of that mixed signal stuff directly.

You can think of the difference between SGMII and 1000BASE-X as which end of the PHY they're looking at on the other end: SGMII expects to see a local PHY on the same board, while 1000BASE-X expects to see a remote PHY at the other end of a long link. Additionally, SGMII supports dropping down to 10Mbps and 100Mbps, based on what the local PHY negotiates to, while 1000BASE-X does not.