2
\$\begingroup\$

I'm doing a thought experiment/napkin design for low cost cluster processor system. Some low cost, high MIPs CPUs chained together on a backplane. My first thought for a backplane is ethernet, because its ubiquitous and fairly easy to implement and run over short/medium distances and has good throughput. I am aware of other backplane technologies, some perhaps better suited but this approach I am trying currently.

Each processor node would be a small credit card sized PCB that slots vertically into a long backplane PCB. The backplane would provide power and whathaveyou to the nodes, and would connect together with a 1GbE link. This would be achieved with a T-bar style 3 port 1GbE switch IC.

          A        B        C
          ^        ^        ^
1GbE IN > X >----> X >----> X > ...

A 1GbE link would feed from one node and into the next, and each node would connect into the main 1GbE link from the switch at 100MbE.

The question is, where can one find a 3-port 1GbE switch IC? I've had a look around at some of the major networking IC companies and they don't seem to exist. I want a low-low cost single chip, not something capable to 5-16 ports. For 100MbE, there are plenty of chips but I think it will be too slow, especially when running in a ring as I intend.

Is there a reason the parts don't exist, i.e. it's a fundamentally flawed concept or it's too niche an application?

What is the likelihood of being able to get an FPGA to do the job, and if so how expensive an FPGA is it likely to need?

I could route a dedicated port for each node on the backplane, and then a master switch at the end; but this will be messy when there are a lot of nodes on a single backplane. Because the nodes are going to be small, I was hoping to get from 16 to 32 of these all connected in a narrow long backplane PCB. 32 nodes would then require 32 * 4 diff pairs, and need more layers for the backplane stackup. The proposed ethernet ring could probably be done on a 2-sided board, perhaps 4 depending on how the impedance control is done.

And to avoid any obvious responses, I am fully aware that running the ethernet ports in a ring will reduce throughput. I am not intending to supply a dedicated 1GbE link, or even 100MbE link to each node. This design is a compromise between speed, complexity and cost. The overriding factor is cost at the moment. I am happy enough with 32 nodes sharing a 1GbE link. The only aspect I am opening up to question is the backplane data link between nodes (and the outside world).

EDIT: Perhaps an ARM core with 2x1GbE ports would be an elegant solution, if in software I could emulate a 3 port switch. Not entirely sure on the feasibility/performance requirements of this.

\$\endgroup\$
  • \$\begingroup\$ The ARM core (and Ethernet solutions in general) are going to be higher latency. Does that matter to you? \$\endgroup\$ – pjc50 Apr 29 '14 at 8:50
  • \$\begingroup\$ It's a sliding compromise. I don't think latency is as important as overall throughput. The issue is whether I can pull out packets destined for a node from the incoming stream push on all the other packets to the next node fast enough. Depends if DMA can handle it all I guess, it its going to slow down 1GbE link then its less than ideal. \$\endgroup\$ – Oliver Apr 29 '14 at 12:20
1
\$\begingroup\$

To create loop topology you will need the appropriate PHY port and chip to support it assuming you can afford the interface with high volume.

E.g. http://www.broadcom.com/products/Physical-Layer/Gigabit-Ethernet-PHYs/BCM5421xE-Family

Consider stub bus topology with a differential 100 Ohm impedance backplane such as Flexbus only does 10MHz while Canbus is only 1MHz. if you want low cost this is it.

Considering the complexity of 1GHz signal, loop topology will have reliability issues if any repeater is bad and any T-bus is limited due to reflections of the stub lengths about 50MHz per metre of bus, this will not be easy unless you have successfully done 100MHz backplanes.

For 1 GHz rates, the PHY supplies the clock and the peripheral must synchronize to it.

\$\endgroup\$
  • \$\begingroup\$ Sorry, I wasn't clear enough - my bad. When I say T-bar, I mean it will effectively look like a multi-drop bus. However it will be comprised of point to point links. It'll just be a load of Ethernet switches connecting in a daisy chain. So there wont be any stubs. \$\endgroup\$ – Oliver Apr 28 '14 at 21:01
  • \$\begingroup\$ Ok I'm an idiot. In connecting up the nodes in a ring shape, and instinctively labeled it a ring topology - which is wrong. Your answer is correct for a real ring topology. What I've actually connected is something different, based on switches - some kind of disturbed star/daisy chain arrangement? \$\endgroup\$ – Oliver May 1 '14 at 9:34
0
\$\begingroup\$

I still haven't managed to find any suitable 1GbE 3 port switches. You can buy a COTS 5 port, 1GbE switch for £10 but can't buy the ICs. Frustrating.

As a compromise solution, I am looking at using a 5 port 100MbE switch at each node, and have 2 x 100MbE links running between nodes.

Also, another option is to use a 5 port switch and connect nodes with a link to it's neighbor and a link to its neighbor+1. Should help reduce the number of hops needed to get from one point to another. enter image description here

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.