2
\$\begingroup\$

I've been asking similar question which received some downvotes, so I'm reformulating my task (hopes this one gives more clarity).

I'm planning to establish a high-speed (3 Gpbs full duplex) serial connection between two fpga boards (distance between the chips < 15 cm, including the connectors), more precisely between spartan 6 and artix 7 devices (these are not absolute requirements, but appear to be the cheapest ones having integrated transceivers inside).

I'm not sure about what protocol should I pick. I've been digging into RapidIO specs lately, but it looks like I have to use a switch chip to connect devices, but switches are pretty expensive (~ $150/piece) and introduce at least 100 ns delay, as well as 10+ layers PCB requirement (example).

PCIExpress is not an option (though connector is nice), since it introduces around 500ns delay, and ideally I'm looking into sub 100ns transmit latency (from parallel stream inside one fpga to parallel stream inside another). Due to high latency 10Gb Ethernet is also not an option.

There's also an Aurora protocol from Xilinx (link), but I cannot find any PHY layer definitions and delay numbers. More of that, despite all high-speed serdes applications advertise low pin counts, all dev boards with embedded transceivers use either SMC, optical or cryptic mezzanine card connectors with 400 pins. Is there any other option? Like, ~50 pin connector or less. So, to sum this up, questions are:

  1. Is it possible to achive sub 100ns delay between two chips using serdes?
  2. Are there any PHY specs/examples for Aurora?
  3. Will it be simpler to throw wide BLVDS interface and use high-clock parallel bus?

Any suggestion is greatly appreciated!

\$\endgroup\$
  • \$\begingroup\$ What's the source of the latency when using PCIe? For protocol purpose, is it unidirectional or bidirectional - do you need lane turnaround or can you have dedicated TX/RX lanes? Could you use a DDR3 interface? \$\endgroup\$ – pjc50 May 29 '18 at 12:17
  • \$\begingroup\$ curious, why so much bandwidth? No Data compression or algorithms? How about a 64bit GPU bus? \$\endgroup\$ – Tony Stewart Sunnyskyguy EE75 May 29 '18 at 12:30
  • 1
    \$\begingroup\$ Key piece of missing information: What is the width of the parallel stream? \$\endgroup\$ – Dave Tweed May 29 '18 at 12:30
  • 1
    \$\begingroup\$ Latency= FIFO memory x T(clk)*packet length + Overhead (can you define any of these?) \$\endgroup\$ – Tony Stewart Sunnyskyguy EE75 May 29 '18 at 12:40
  • 1
    \$\begingroup\$ 3Gps/100ns ~ 3kbit frame + 0 Overhead or ? 24kbit frame/8channels + 0 OH or 192kbit frame/64bit channel + 0 OH or? What is your frame buffer size? \$\endgroup\$ – Tony Stewart Sunnyskyguy EE75 May 29 '18 at 12:50
2
\$\begingroup\$

Aurora looks like a reasonable choice; basically it looks like the thing you'd build if you were trying to build a generic interface from high-speed SERDES. It has flow control. It even has channel bonding, which seems to have some support in Xilinx hardware. That should give you your required speed from theoretically a single link pair in each direction at the 3Gpbs max.

This link gives some latency numbers, for a loopback link. Board to board latency will obviously depend on wire delay.

It shouldn't require a more complex stackup than the FPGA itself (usually minimum 4?). I would suggest using the PCIe connectors even if you don't use the protocol; alternatively SATA connectors might provide a good short cable option.

They don't actually specify the PHY at all, but this app note gives an example of how to connect two eval boards. They're just wired up with mini coax leads using SMA connectors, so I guess the required impedance is the usual 50 ohm?

\$\endgroup\$
  • \$\begingroup\$ Thanks! But why did you specify that I will need minimum 4 FPGAs? Do you mean transceiver modules/channels? \$\endgroup\$ – user37741 May 30 '18 at 14:14
  • \$\begingroup\$ Sorry, minimum 4 layers in the PCB. \$\endgroup\$ – pjc50 May 30 '18 at 15:28
0
\$\begingroup\$

You should definitely use the gigabit transceivers. With Spartan 6 you can send up to 3.125 Gbps using one transceiver so that is no problem.

These transceiver works very well and are very easy to implement.

If you have SFP connectors on both boards you only have to buy one SFP copper wire cable to connect the two boards. If you have SMA you only need to buy four SMA cables. There are also SFP to SMA cables.

When I have done these kind of project I usually define my own protocol however you need to send some K-words to align the bytes in the serial stream. You also need to define an idle sequence for the elastic buffer. Usually I use the same K/D as defined for Ethernet.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.