0
\$\begingroup\$

An FPGA endpoint needs to needs to transfer large quantities of data to the host processor's RAM over DMA. So memory mapping to a single RAM might not be possible. Normally high end PCs or Servers have multiple RAM slots instead of a single one.

  1. Can the FPGA transfer memory to the multiple RAM slots? If one RAM is full, can the FPGA transfer data to the next RAM? If yes how is this switching done by the PC Root Complex?

  2. Does the chipset attached to the PC's processor have multiple memory controllers(or MCHs)? Can the processor access one RAM and keep one for FPGA DMA?

Any advice on this is appreciated. Thanks.

\$\endgroup\$
2
\$\begingroup\$

PCIe can DMA anywhere within the system physical address space, even outside of the local NUMA node, though there will be a performance penalty associated with that. This is all handled exactly the same way the CPU can read or write wherever it likes without needing to know what physical device those addresses correspond to. Accesses from different sources will be time interleaved.

Memory 'filling up' is a higher level concept, this would need to be managed by pointing the DMA engine on the PCIe device to free sections of memory. Generally this is done from within an application or device driver.

\$\endgroup\$
  • \$\begingroup\$ The only caveat: a single PCIe transfer may not cross a page boundary, so all of the data will go to the same destination. \$\endgroup\$ – Simon Richter Jul 27 '17 at 15:15
  • \$\begingroup\$ For instance servers have multiple RAM slots like 24 in case of HP ProLiant ML350. The memory controller in the processor(Xeon) usually supports Quad Channel mode. So how are the 24 DRAMs accessed by the PC? What is the architecture of the memory bus? \$\endgroup\$ – electro_sm11 Jul 31 '17 at 10:02
  • \$\begingroup\$ The memory will be 'mapped' into physical address space, likely with various forms if interleaving to maximize performance - a read of a few MB of contiguous physical memory may hit all memory controllers and DIMMs in the NUMA node (socket). \$\endgroup\$ – alex.forencich Jul 31 '17 at 17:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.