3
\$\begingroup\$

I'm reading on 100G Ethernet and came across this article by ADVA. On page 2, top paragraph (before "Spectral Efficiency"), it compared link-aggregated 10x10GbE and a "true" 100GbE and says

A true 100GbE path between two core routers will produce 100Gbit/s of throughput, more than twice the real performance of a 10x10GbE aggregated link.

But what is the real difference between them? As I understand the "true" 100GbE also put 100G on 10 different wavelengths each carrying 10Gb/s (some cases 4x25G), how is that not link-aggregation? Is the difference not in the physical layer but how you code the packet onto the physical layer? Is there a reference how this is done?

\$\endgroup\$
7
  • \$\begingroup\$ I imagine it has to do with the overhead of the individual virtual links. \$\endgroup\$ Jan 3, 2013 at 17:16
  • \$\begingroup\$ @TobyLawrence, That's the part I don't understand, what's the difference in the overhead? What is virtual link? I know more Physics than "Ethernet". \$\endgroup\$
    – LWZ
    Jan 3, 2013 at 17:30
  • \$\begingroup\$ @LWZ, most Link-arrgregated connections need to have software to both split, then join, all the streams. This adds processing overhead, bogs down the routers. That is what a virtual link is, the process of deciding ten hardware links should be treated like one. A true 100gb connection can do more of it, in hardware, leaving the router's cpus free of the extra work. \$\endgroup\$
    – Passerby
    Jan 3, 2013 at 17:33
  • \$\begingroup\$ @passerby, that makes sense, but the thing is, a true 100GbE also has 10 hardware links, namely 10 transmitter/receiver on 10 wavelengths, doesn't it? You don't have any 100G electronics for a "true" 100G link, right? And the 10 wavelengths are treated as one, so the processing has to be done somewhere, if it's not router's cpu. \$\endgroup\$
    – LWZ
    Jan 3, 2013 at 17:59
  • 1
    \$\begingroup\$ @lwz The PCS allocates traffic into the individual lanes and leverages the same 64B/66B encoding used in 10ge, where each 66-bit word is distributed in a round robin basis into the individual lanes (See Figure 63). The lanes are then fed to the PHY for transmission through the appropriate medium. For a 40 GbE circuit, the PCS would divide the bit stream that has been passed down from the MAC controller into 4 PCS Lanes each consisting of 10 Gbps streams. These Lanes are then handed down to the Physical Medium Attachment. PMA layer contains functions for transmission and reception of the lanes. \$\endgroup\$
    – Passerby
    Jan 3, 2013 at 18:44

2 Answers 2

2
\$\begingroup\$

Extreme Networks, a networking equipment manufacturer gives a good explanation of 40gbe and 100gbe links, which build upon 10gbe standards

In essence, Link-Aggregated 100gbe connections need to make up in software what a dedicated 100gbe can do in hardware, including error checking and data splitting/joining.

\$\endgroup\$
0
\$\begingroup\$

Some protocols (earlier versions of TCP for example) don't like it if packets arrive out of order. While the internet doesn't absolutely guarantee in-order delivery, having packets arrive out of order is considered a bad thing.

That is a problem when you start trying to aggregate links. It is very likely that due to slight differences in cable lengths and processing delays That different links in an aggregation group will have different delays.

If a switch simply took packets from the queues and sent them down whichever link in the aggregation group was free then there is a very real chance that slight delay variations would cause packets to arrive out of order at the destination.

To avoid this link aggregation doesn't send packets down whichever link is free. Instead it distributes the frames in a way that maintains packet order for any given "pair of terminals". Typically this involves creating a psuedorandom number based on the source and destination MAC address. So each source/destination MAC pair uses only a single link from the aggregation group and hence packets for any given source/destination pair are delivered in-order.

http://www.admin-magazine.com/Articles/Increasing-Throughput-with-Link-Aggregation

The problem is this means that the traffic can spread unevenly among the links, especially if the number of source/destination MAC pairs active at a given time is fairly low This uneven spread can lead to one link in the aggregation group maxing out while there is still spare capacity on other links.

While a 100 gigabit link does consist of multiple optical transceivers at the hardware level they are tied together at a very low level with very well defined timing. So it is no problem to ensure that a single stream of packets is delivered in the correct order.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.