One criticism of GENEVE (a network encapsulation protocol) is that it uses tag-length-value fields, and these are hard to process in hardware.

Why is this considered hardware-unfriendly? What approaches would be more friendly to high speed hardware implementation, and why?


2 Answers 2


Tag(or type)-length-value (TLV) is a method of containing different types of info of variable length in a data structure.

You only need that when there's no common subset of info that every instance of that data structure must have, or when the order of fields must be variable for some reason.

Think about it this way: if all GENEVE packets had a "destination" field that contained say a 64bit network address, why not simply define that every packet starts with 64bit of destination address, and save yourself one tag and one length field?

Hardware (and that includes hardware that runs software!) is good at looking up values at specific positions. So, parsing a header where the destination address is always at position x and always has length y is very easy; you just read memory address x and interpret the result as integer of length y. That's what e.g. CPUs do for every single thing they read from RAM.

If you, on the other hand, to understand a packet, first need to look for specific fields by going through TLV fields (ok, first field says it's not the tag I'm looking for, has length 14, so jump ahead 14 bits, aha, not the field I'm looking for, jump ahead,…) then parsing that packet will be slow. That counts for software just as much as for hardware implementations.

Even worse than slow, it'll be non-deterministic in complexity. So, some TLV structures might take only a single clock cycle to analyze, others need to iterate through 42 fields to do the same. If you're implementing a signal processing application, accounting for random delays in one of the steps quickly becomes a nightmare, as you suddenly need to buffer input, or apply backpressure, or drop data, just because someone decided to have a flexible data structure.

In software, it's often cheap and relatively fast to just preallocate a header structure with fixed offsets and "fill in" these fields as you iterate through the incoming TLV structure. But: for that, you need RAM, and often quite a lot of RAM, if you can't know which fields you're looking for the moment you start parsing the structure.

So, TLV is a common scheme for serialization of weakly structured data for permanent storage or slow transmission. It's usually pretty undesirable for streaming applications, where the same kind of data comes by pretty often (e.g. network packets, video frames, infrastructure operation commands…); in that case, you'd much rather pre-define fixed structures, even if that wastes a bit of transport bandwidth for occasionally unused fields.

For example, most systems don't use all the fields an Ethernet packet can have. You still wouldn't try to save two bytes – transporting 1490 or 1492 bytes on Gigabit ethernet doesn't make much difference, but having to check for every single packet whether your packet is of type A or type B does have a negative impact.

@Janka raises an important point: Assume your hardware's whole job is just to copy the whole packet from in- to output. Now, great, instead of telling your DMA engine to copy one packet worth of data from in- to output, you're parsing all input to figure out how long your data is. That is way, way slower than just copying data.

  • 3
    \$\begingroup\$ +1, but the very short answer is: DMA is not going to parse the data, so length fields do nothing but adding overhead. \$\endgroup\$
    – Janka
    Oct 7, 2018 at 17:53
  • \$\begingroup\$ Well, @Janka, yeah, that's the short form of "where others can just pick a fixed position and length of data, the TLV user has to parse the full structure first". \$\endgroup\$ Oct 7, 2018 at 17:56
  • 1
    \$\begingroup\$ This is all true, but compared to just about any other pattern allowing variable length or optional data, TLV is efficient, easy to process, and permits future extension. On a scale of ethernet headers to SQL to "date on a renewal sticker on a license plate captured in a surveillance video", TLV lands far closer to the more efficient/machine-friendly end of data transmission. \$\endgroup\$
    – Ben Voigt
    Oct 8, 2018 at 3:44
  • \$\begingroup\$ The only TLV I know of is stuff like Gzip or any variable compression file. Not the same as normal data or even encrypted data. \$\endgroup\$
    – user105652
    Oct 8, 2018 at 4:34
  • 1
    \$\begingroup\$ @MarcusMüller: Well, in ASN.1 some data have fixed record structure and minimize embedded tags in some encodings, but there's also a lot of use of unions, and any time a union or sequence thereof appears, you get more TLV. And others of the encodings do tag everything. \$\endgroup\$
    – Ben Voigt
    Oct 8, 2018 at 6:07

In addition to Marcus' great answer, I'd just add that length prefixed fields are often considered inferior to terminated fields because they require an extra counter to process.

The classic example is C strings. Only a single pointer is required to process a string because you just keep incrementing the pointer until you reach the null termination character. With a length prefix you need an additional counter to track how many characters you have processed, which means more memory and more processing overhead to keep incrementing it.

It may seem like a small thing, but on very low power or low cost or low spec systems it matters.

  • \$\begingroup\$ OTOH, code which processes C strings often needs to scan the string once to find the length, then again to process it. You win some, you lose some. \$\endgroup\$ Oct 8, 2018 at 9:24
  • \$\begingroup\$ That's the other part of the design that needs to be carefully considered, to avoid the need for doing that. \$\endgroup\$
    – user
    Oct 8, 2018 at 9:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.