5
\$\begingroup\$

Modern SoC solutions (FPGA+ARM) can exist both as a PCIe board, but can also work disconnected (with a 19V PSU). So my question is - is there a solution which provides just a PCIe motherboard/bus, without the typical trappings of home PCs (such as CPU socket, RAM slots, SATA and all that)?

Basically, what I'm looking for is simply a general-purpose PCIe fabric that can connect to a power supply and provide power and communication capabilities to PCIe cards. In terms of the cards I'm interested, I'm talking mainly about SoC accelerators, GPUs and Xeon Phi.

P.S.: yes, I know this can be built on top of existing COTS motherboards, I'm wondering specifically if there are headless solution, and whether it is even feasible. Thanks!

\$\endgroup\$
  • 3
    \$\begingroup\$ You mean like an industrial PCIe backplane? \$\endgroup\$ – Ignacio Vazquez-Abrams Nov 5 '16 at 10:05
  • 3
    \$\begingroup\$ No offense, but a) as @IgnacioVazquez-Abrams said, a google for "PCIe backplane" would have solved that, and b) in reality, you'd typically would want some kind of CPU to be present to at least set up everything else (A GPU will do exactly nothing without some kind of OS, and even if it's just good ol' BIOS, supplying data and configuration (how would you run CUDA without a PC-thing?); a Phi will not know what to do until someone instructs it to, and whatever your SoC does...). But yes, PCIe is point-to-point, and you can have a bus wherever you want. \$\endgroup\$ – Marcus Müller Nov 5 '16 at 10:09
10
\$\begingroup\$

All PCIe buses must have a "Root Complex" (RC) which is what configures and enumerates all devices. Whether that is a desktop PC, and FPGA, or some other SoC device is entirely up to you as long as it provides the required functionality.

You can get many SoC devices, including ARM SoC/FPGA hybrids which will allow you to provide RC functionality in a compact form. The thing to remember though is that once you have enumerated the bus - once the RC has detected all cards, assigned addresses, etc. - you still need to be able to interface with whatever device you are using. That typically means a computer running an OS that has the required drivers for the device.

Many of these SoC devices can run fully fledged Linux installations which would, assuming there are Linux drivers for your device, be able to host the PCIe bus and control all of the devices - interface with compute modules, GPUs, etc.


Of course you don't always need an OS running to use PCIe. PCIe is just a generic communication protocol. Once you have enumerated the bus with your RC, how you communicate between two devices is entirely up to you if you are designing the devices yourself. For example you could set up two or more FPGA devices on a backplane with one as the RC and others as endpoints. You can then program your FPGA however you like.

If configured with bus master capability during enumeration they can talk to other devices, transfer data, send interrupts, anything they like. But more importantly, they can do it all with complete autonomy - no need for a computer or operating system to instruct them. The only requirement is that there is some defined structure for how this is done - the other cards have to understand what is being said to them and know how to react. You can build systems of FPGAs talking to each other and transferring data using pure logic circuitry instead of an OS.

This is not just limited to FPGAs. You can include GPUs and other devices in your bus. However in order to do so, you need to know how to communicate low-level with the device. You need to know what registers do what, what sequences of commands to send, and so forth. For a device like an FPGA that you have programmed yourself, that is fine, but for things like compute modules and GPUs, this low level information is not readily available, and you wouldn't have the luxury of using the drivers that the device maker have written for mainstream OSes.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.