6
\$\begingroup\$

I'm looking to build a simple board that has a 16x PCIe connector, that seats a compatible PCIe video card.

I can find the connectors and create a footprint, on the board. I can find and understand to some degree the basic pin-out of the PCIe 1x connector -- but I'm not clear on the full end to end implementation.

I've read the full specifications costs lots of money, and honestly is overkill for what I want.

I'm not super concerned about the actual protocol (like at the physical layer), so not necessarily looking for the official PCIe spec -- just the nuts and bolts implementation. Which pins go where on the cards, where they are laid out.

I am not certain on signal routing both between the mating connector and the PCIe card, and then where the card signals.

I know Altium provides templates of the actual pads and so on, but I'm more interested in where the signals go, and what each one does, and which signals need to be treated as high-speed signals and so on.

It seems like this kind of information isn't so open-source? It's not like a video manufacturer is just posting up a datasheet for every person who buys a PCIe card to understand how it is routed from card to the eventual CPU on a motherboard.

For people who are building video cards or whatever for PCIe, where do they learn how to build these devices?

\$\endgroup\$
2
  • 1
    \$\begingroup\$ What are you putting at the other end of your board with a connector? \$\endgroup\$ Commented Sep 6, 2017 at 21:11
  • 1
    \$\begingroup\$ Well as a learning project for high speed layout, this will be like a GPU riser for bitcoin mining. Basically the board would be a regulated power source, a PCIe 16x connector, a ribbon cable that goes to a separate PCB with a PCIe 1x connector. If you were building a motherboard, do you interface the PCIe card to a controller? What is that controller called? (Literally, I don't know where the start of the thread begins). \$\endgroup\$
    – Leroy105
    Commented Sep 6, 2017 at 21:15

2 Answers 2

10
\$\begingroup\$

PCIe has "End Points (EP)" and a "Root Complex (RC)". The GFX card in this case is known as an EP which is basically like a leaf of the tree. At the trunk of the tree would be the CPU on your motherboard which has an RC built in to it which manages all devices.

Wiring between an EP and an RC is at it's simplest just point to point. Within this there are two types of signals - lanes (TX and RX), and sideband signals (everything else, e.g. clock, reset, power, etc.). Of these signals the following should be noted:

  1. For the sideband signals, these should pretty much all just be connected between corresponding pins at either end. In your case for a riser card you'd simply pass them straight from the pin on one connector to the same pin on the other.

  2. Power traces should be routed with sufficient with to supply either 25W or 75W. There is generally a 12V supply, a 3.3V supply, and an auxiliary (standby) 3.3V supply when the PC is asleep or powered off.

  3. The lane signals are the most critical. These must be routed with great attention to detail ensuring they are differentially routed and impedance matched to 100 Ω differential. This is typically achieved by using a ground plane on a 4-layer board as a reference plane (HF routing is beyond the scope of this answer, there are plenty of EE.SE resources here). You must length match the differential pairs P/N signals for each lane, however you don't need to worry about matching the lengths of different lane for PCIe as each lane is handled separately from the rest anyway.

  4. Lanes are unidirectional - i.e. the data flows one way. It is imperative to cross-over TX and RX lanes - the TX lane of one device goes to the RX lane of the other device. However, if you are building a riser card, the cross-over of the lanes is done on the motherboard, which means that your riser should routed the signals from one connector to the same pin number on the other end (TX on one end goes to TX on the other end).

  5. If you are changing the number of lanes, you need only wire up the lanes you require ensuring that the lowest number contiguous chunk of lanes (e.g. Lane 0 for 1x, Lanes 0 & 1 for 2x, etc.) are kept. PCIe expects that lane 0 on one device connects to lane 0 on the other device always (*) and that no lanes are skipped. So if you are connecting your x16 card to a x1 slot you must keep lane 0 and need only wire that one up. Any unused lanes can be simply left disconnected (you don't need to nor should you terminate them).

Beyond that it is just a case of following good PCB practice and taking special care with impedance matching and high frequency trace routing.

The part which gets very tricky is if you hope to get Gen 3 PCIe (and Gen 2 to a certain extent). The speeds at which G3 runs (4GHz) are far beyond the capabilities of standard FR4 PCB substrates and routing them takes you into the weird and wonderful world of RF design - think Wi-Fi radio signals being channelled through PCB traces and you realise how much of a pain it is. Gen 1 is only (o_O) 1.25GHz which is a little bit more forgiving as long as you are neat with your routing and try to avoid vias where possible.


(*) Strictly speaking there are other clever tricks like lane reversal but support is not mandated.


With regards to the final question of where do people learn this stuff - generally while working for companies or research institutions. There aren't a lot of resources out there because designing PCIe devices is generally far beyond what most hobbyists would attempt, and the implementation information is probably kept as corporate secrets by those who make commercial devices.

\$\endgroup\$
1
  • \$\begingroup\$ Great answer, thank you! Yes, the PCIe device design is a very hush-hush thing. I'm trying to implement communication lines, not build a video card! I've routed and built sub-GHZ RF boards for different systems (50Ω traces, with a matching network, tune it with a VNA). But never done a high-speed board. This seems like a good learner project. \$\endgroup\$
    – Leroy105
    Commented Sep 6, 2017 at 21:50
4
\$\begingroup\$

You definitely need a copy of PCI Express Technology 3.0 (2012), or you might get by with PCI Express System Architecture (2004). Both these are Mindshare books and cover the basics very well.

Intel also have great information on their design network. They also have a version of the PCIe 4.x spec freely available.

If you are only doing PCB layout, then get a copy of the PCIe Board Design Guidelines. You have to sign up with Intel to get the latest version, but there are plenty of copies of the draft available.

\$\endgroup\$
1
  • 1
    \$\begingroup\$ Thanks, I haven't seen a reference to that 2012 source. Kind of mixed reviews on the 2004 text. I worried it is a little down in the weeds, physical layer implementation. This is more of a plumbing project hooking up pipes. Intel guide seems great, i will check it. \$\endgroup\$
    – Leroy105
    Commented Sep 6, 2017 at 23:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.