How to make an external PCI adapter that runs on USB3.0?

Question

Motivated by old PCI (NOT PCIe) cards, and laptops without expansion slots, I want to try making a PCI-over-USB 3.0 solution. This is about 70% experimentation, and 30% actual use.

Is this viable? I was thinking of buying something like this Cypress USB controller, in conjunction with a PCI bus master board, and (I'm guessing) a microcontroller to interface the two.

I'm not experienced in this, which is why I want to learn... I am an EE, but not this advanced.

Is this doable? Would it be possible as a long term learn-and-do project?

Answer 1

There are two challenges:

1. Latency

   USB has two orders of magnitude higher latency than PCI. Many PCI drivers assume memory mapped access to registers on the PCI card with the CPU blocking on the bus transaction. Unmodified PCI drivers over a PCI-over-USB bridge are likely to fail because of wrong latency assumptions.

   Check this study on PCI latency: PCI mean latency is in the order of 400ns, PCIe mean latency is in the order of 2000ns. USB 3.0 latency according to the article you linked is in the order of 30us.

2. Bus vs. Packet Network

   While PCI is a bus, PCIe and USB are packet based networks. The PCIe root bridge hides the complexity of a packet network and presents the CPU with something that looks like a bus. Doing the same thing over USB is certainly possible, but if you want binary compatible drivers your hardware interface towards the CPU has to look more or less like PCI and you will end up with something like a PCIe root bridge. This is probably not something that can be achieved by common USB chipsets with *HCI interfaces.

Answer 2

PCI cards are enumerated and assigned addresses in the host's address spaces (both memory space and IO space). The cards can DMA to anywhere in main memory and in principle even DMA to each other bypassing main memory (though this is rarely done outside of specialist kit from the likes of National Instruments).

Drivers for PCI cards don't expect to pass requests through lower level drivers. They expect to be able to talk directly to the card through the memory and/or IO map.

You can try and fake memory accesses, for example by using the virtualisation features of the host CPU but you are likely to run into significant latency issues.

I therefore find it unlikely that such an adaptor can be made to work acceptable with the existing drivers for the cards. The only viable way to make such an adaptor would likely to be to re-engineer the drivers for each card you want to work with it, carefully deciding what tasks to keep on the host computer and what tasks to move into the adapter.

If you want to use your hardware with the standard drivers then you will need an appropriate PCI-like interface to the host, such as cardbus, expresscard or thunderbolt.

Answer 3

I don't see any reason it isn't doable, but you're taking on quite the project here. There are a lot of layers you'll have to implement to make it successful. Your final solution will probably look something like this:

PCI Physical -> PCI controller -> MCU translator -> USB client controller -> USB Host Driver and USB to PCI device emulation

I think your proposed approach is probably the most simple, because you would only have to program the man-in-the-middle microcontroller to be a translator between the PCI bus master and the USB controller (and the host driver). However, I like to combine hardware components wherever possible, so here are a few alternatives for you:

• System on a Chip (combines PCI Phy & bus master, MCU, and USB controller)
  System on a chip solutions, similar to the Cypress PSoC,could allow you to reduce extra hardware by integrating the PCI PHY layer, PCI controller, and USB controller on your device into a single chip, while still retaining the ability to do some of your work on a CPU in C instead of all in an HDL. As you have pointed out in a comment, the bandwidth of the PCI bus (133MB/s at its slowest) is far greater than FSUSB provided by Cypress for PSoC chips(~1.5MB/s), which will create bandwidth issues. That particular chip won't work for you, but I chose it as an example of a SoC since it's the only one I have worked with. You will need to find one with USB 3.0 support to meed your bandwidth requirements.

• All FPGA (combines PCI Phy & bus master, MCU, and USB controller)
  You could possibly fit all of the PCI and USB work onto a single FPGA if you're feeling confident in your HDL skills. Try looking for open-source HDL implementations of the PCI and USB protocols to save yourself some work.

• Microcontroller w/ USB 3.0 (combines MCU and USB controller)
  A third option would be to use a chip like TI's Sitara MCUs that contain a USB 3.0 controller and PHY, which could be connected to your PCI bus master instead of having a separate MCU. Googling for "USB 3.0 microcontroller" shows a few other options.

Unfortunately, all of these alternatives still require a driver on the host side that can make a USB device appear as a PCI device so that something can actually use it. I don't know of any libraries that can help with this, so you may be on your own for that component.

Answer 4

The existing product is made by ARS Technologies (arstech.com). Their web site suggests that they are US based, but I don't think they are.

I don't recall if the host permitted use of existing PCI drivers or if it required a USB driver interface.

Answer 5

Note: responding to an old thread. This might not what you want to hear, but these cards can be bought on Taobao in China, and therefore also very likely on Aliexpress. These cards will show up as an PCI Bus and PCI to PCI bridge. I own several for us with older hardware cards (like the Catweasel MK4)