Why not implement 1Gbps, when all I need is 20Mbps?

Question

Background

I am working with a client on a large project which requires a custom networking chip to be designed to solve the data transfer requirements within the project. The network is intended to send small packets a few inches from one PCB to another over a single twisted pair cable. We will be designing and specifying the network protocol, and another company will be responsible for the silicon implementation.

I estimate that 20Mbps data rate between nodes will easily cope with the amount of data that needs to be sent, with plenty of head room should the amount of data increase in the future.

Problem

The client is asking me why I am specifying only 20Mbps. Why not something like 1Gbps? Wouldn't that be better? Intuitively, I feel that cranking up the data rate massively beyond what would be needed is a bad idea. Initially, I thought that the cabling would need to be shielded (which I don't want), but looking at the Ethernet cable categories, I see that Gigabit Ethernet can run on Cat 6 cable, which doesn't need to be shielded.

Other Constraints

• The project is desperately space constrained, and do we don't have room for things like magnetics, unless it's a very small component (0603 max).
• The cables need to be as slim and flexible as possible.
• The device will run from plug-in power, so there is no particular low-power requirement.

Question

What are the problems, in terms of silicon design, cabling, and anything else, that may be faced at 1Gbps, that wouldn't be nearly as bad at 20Mbps? Should I go with my client's suggestion of implementing the network at 1Gbps, or should I insist on implementing only what is required?

We're under strict NDA, so I can't give too many details about our requirements. But please leave a comment if clarification is needed.

Answer 1

A few reasons:

Power

Faster speed means more power. Not only do you need faster analog circuits, which will consumer more power, all your electronics surrounding them need to be faster. Your digital systems, your latches, clock management, etc. If you get that 1 Gbps by using multilevel signalling you now need better ADCs and DACs. You might need to start dealing with more complex filtering. You could start requiring FEC which also needs to keep up.

Chip size

Faster means more going on. You need better clock stability, which means bigger circuits. You need better timing, which means a more complex clock recovery system. You might need to switch to using DSP to do channel equalization. Your potentially needed FEC needs chipspace.

Environment sensitivity

If you switch from a few tens of megabaud to whatever is needed for gigabit, you will become far more sensitive to the environment. Small mismatches which might be unnoticeable at a few tens of MHz become resonant stubs at higher frequencies. Reflections might start causing intermittent performance. A nicked cable due to abuse over the years (I don't know the application environment for your product) might be fine for lower speeds, but cause poor performance when you go higher.

Design effort

I think it is obvious from all of the additional issues I discussed above that the time and effort of designing a faster communication link is significant. This alone should be enough of a reason.

EMI

Faster speed means meeting EMI requirements could be harder.

Answer 2

TTL (single-ended, unterminated) signals can easily handle 20 Mbps or more — look at SPI, for example. If you're only going a few inches, ribbon cable and IDC connectors (or a backplane of some sort) will get you from board to board.

1 Gbps puts you into the realm of having to deal with impedance-controlled traces, connectors and cables. The receivers will need to use PLL/DLL techniques to maintain synchronization and separate clock/data, whereas at the slower speed, normal synchronous logic will be sufficient. The 50× overkill and the additional headaches are simply not worth it, if you're sure that 20 Mbps will suffice for the foreseeable future.

---

I once designed (25 or so years ago) a custom serial bus protocol for board-to-board control and status among boards in a telecom rack. Sort of a cross between I²C and SPI — unidirectional signals like SPI, but embedded device addresses like I²C.

Answer 3

The obvious question is, "Does 1 Gbps mean 1000BASET Ethernet?" If that's what the customer is thinking, your requirement that, "we don't have room for things like magnetics" rules that out right away. Ethernet does use magnetics on the physical layer, and when I designed an interface some years ago the magnetics were part of a roughly 1 inch cube.

You say you're using FPGAs, but you don't say whose. If you're going with Xilinx, you should be aware that the current models natively support LVDS, which would seem ideal for your purpose. Early LVDS systems (hi-def televisions) ran at 122 Mbps, and the technology can go well over a Gbps if you really need to. Being differential, and assuming your two boards are not using wildly floating grounds, noise immunity is excellent.

As for your specific choice of clock frequencies, adding more headroom than you think you need is one of those decisions which can save your bacon in the future, so I wouldn't rule out picking something like 100 MHz, but that's up to you. You might acquaint your customer with Roberge's Law (Jim Roberge was a well-known electrical engineering professor at MIT a few decades ago): "Those who ask for more bandwidth than they need deserve what they get." Granted, he was talking about servo systems, but the principle remains good over a remarkably wide range of disciplines.

Answer 4

The application that you describe does not make sense to jump right into a custom silicon solution. The data rates that you anticipate can be easily handled by moderately priced FPGA technology and the FPGA can be programmed to implement the special protocol if your really believe that such protocol is needed.

Much more often you should be considering a standard physical layer and then build the customized protocol on top of that. For a net comm channel bandwidth of 20Mbps you should plan for a protocol overhead of some amount because framing, error check coding and synchronization eat up some of your bandwidth. So maybe consider a higher raw bandwidth to accommodate this overhead.

Once you have your design proven out then you can go to the FPGA vendor and get them to produce a hard chip design from the FPGA programming. This approach mitigates all early development risk and lowers overall NRE costs a tremendous amount over the "dive into custom silicon just because it seems cool".

Answer 5

The actual question is, why to design a protocol when everything already exists.

For Ethenet solutions you take 10/100 and not 1GbE because it's still a little cheaper and much easier to layout. By the way, Ethernet can work without magnetics. But it does require MAC, which can be extra IC. Or do you have one in a microcontroller?

20Mbps is something that fits Rs485 or such layer, which is even cheaper and simpler. Twisted pairs come with all kinds of cables, more or less flexible, with connectors or just soldered to your PCB.

Ah, most importantly. It's easier to screw up with 1Gb. But if they need room for further growth, it limits less.

Bottom line: you need to understand your system requirements.

Answer 6

I'd suggest the simplest route with the best likelihood of success and least software overhead would be to implement a 100Mbps Ethernet connection. You can implement this without any magnetics involved when the distances are small.

Here's a start with information on the Intel 8255 PCI-Ethernet controller, and an application note on connections without the magnetics.
I'm not suggesting you use the 8255, but you can get IP (10/100/1000Mbps) for any of the FPGAs you are likely to use very easily, and it's well debugged.

Assuming you have a processor in the mix, supporting a standard Ethernet controller is a very low software effort way to implement point to point networking.
We used a bunch of this type of point to point connections on specialized motherboards at Intel, they were easy to debug, and very reliable.

Answer 7

The answers here are technical, i give a requirements engineering perspective:

My view on it is simple

• You need at least 20Mbps to make it work, so dont specify 20 but "20 or more" for the application.

• any faster hardware also fulfills your requirement

• if the faster HW is cheaper/easier to develop because of existing standards, then your requirement can be fulfilled by these ones, too.

• If the customer wants more, try to figure out if there is something behind it (it could be that they already plan upgrades and want to stay compatible between boards when swapping)

Answer 8

Power, signal integrity, and timing. I worked on a chip with a 25 gbps interface and that meant a clock rate of 1.6 GHz and a ton of power. If we could have run at 19.2 the clock rate would have been 1.2 GHz. Greater than 200ps of extra margin per clock period, that would have been a huge help.

I've never done board design but I expect that 20 Mbps is no problem. 1 Gbps is still not that tough but a lot harder than 20 Mbps.