0
\$\begingroup\$

PCIe offers a very high speed interface between a peripheral card and PC. Sending and receiving data from the FPGA side which contains the PCIe IP seems to make sense. However, I am confused how it would work on the computer side. On the PC side we have different operating systems and they usually do not allow direct access to I/O ports.

How would one write a program on the PC side (lets say windows) to communicate with the PCIe peripheral card? I think most likely we would use C or C++ but don't know anything more than that.

\$\endgroup\$
1
\$\begingroup\$

If you use PCIe on an x86 system, you will need DMA in the endpoint. Otherwise your transfers will be quite slow, as x86 platforms don't tend to have host-side DMA resources. Reads, especially will be painful as they're non-posted and take a long overhead for each transfer.

In Xilinx-land, the most popular PCIe bridge block is called 'xdma', which is documented in PG195. This block supports memory-mapped or streaming DMA, and has drivers available for Linux and Windows. While it's not terribly complicated, you do need some kernel expertise to integrate it with your application. It's a well-known block, and there's lots of chatter on the Xilinx forums about it.

There's a couple of other IP blocks in Xilinx (like cdma and vdma), or you can purchase a block as IP from companies like Northwest Logic.

As far as how PCIe devices 'work', they are enumerated at boot time using I/O, and configured by the driver to live in a specific set of memory address spaces, called 'resources' in Linux and BARs in PCIe. The way you communicate with a device is you ask for its virtual address when you 'open' it, then you can read/write to that address. That address isn't fixed: it changes as it's assigned by the MMU each time you open the device.

If you want to see how that works, the utility 'pcimem' (Linux) shows a very basic way to do that. It's a simple program that does peek/poke to any PCIe device.

\$\endgroup\$
4
\$\begingroup\$

You just found out you need a driver to communicate with hardware. Indeed!

And that driver will depend on what you do on the FPGA. Your FPGA vendor provides examples for examples of things you can do!

\$\endgroup\$
2
  • \$\begingroup\$ Driver writing is an esoteric art, something with little information out there on as a subject. \$\endgroup\$ – Quantum0xE7 Aug 25 '20 at 1:12
  • \$\begingroup\$ @Quantum0xE7 that's not true. While I can't say FPGA vendor driver code is the most beautiful in the world, they do ship it and you can look at it. \$\endgroup\$ – Marcus Müller Aug 25 '20 at 7:00
2
\$\begingroup\$

You would either need to create a device driver or use/modify an existing driver. It also depends on whether you use the "bare" PCIe IP core, or something slightly higher-level such as RIFFA, XDMA, or QDMA. If you use the bare PCIe IP core, you'll probably want to write your own driver. If you use something higher-level, it may come with its own driver, which you can then interface with from your own application software.

\$\endgroup\$
6
  • \$\begingroup\$ I think that driver writing is perhaps the most obscure subject I have come across in the world of electronics, I am talking about drivers for things in Windows. \$\endgroup\$ – Quantum0xE7 Aug 25 '20 at 1:23
  • \$\begingroup\$ Simple solution: don't use windows, instead use linux, where you can at least read over the source code of existing drivers, and the OS components that they interface with. \$\endgroup\$ – alex.forencich Aug 25 '20 at 1:27
  • \$\begingroup\$ I see, but when using Linux, there is one 1 Linux, there are different variations. Does it matter which is chosen? Maybe some flavour is easier than others to be used for something like this. \$\endgroup\$ – Quantum0xE7 Aug 25 '20 at 1:35
  • \$\begingroup\$ All the distros run the same kernel, although some run newer kernels than others. Now, the kernel is a bit of a moving target and the APIs change a bit over time, but you can track this sort of thing using things like elixir.bootlin.com/linux/latest/source. \$\endgroup\$ – alex.forencich Aug 25 '20 at 1:38
  • \$\begingroup\$ aha, the Kernel is the same? hmm, where can I find out more about this? Does this mean that Linux running on NASA space probes, supercomputers, our mobile phone and the desktop all has the same Kernel? \$\endgroup\$ – Quantum0xE7 Aug 25 '20 at 13:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.