0
\$\begingroup\$

I am designing the simplest form of a camera: an FPGA that interfaces an image sensor and sends the acquired data to the host device via USB 3.0. There is no processing on the FPGA, it is used for sensor control and data acquisition.

By studying various resources online, I observed that typically a DRAM is included in the design to store sequential frames as well as intermediate processing results. At first, my plan was to implement:

Image Sensor ==> FPGA ==> DRAM (Acquisition)

DRAM==>FPGA==>USB Controller (Transfer to host)

Since no image processing is done, I was wondering if it is safe to omit the DRAM in the design for reducing design complexity. So, the pixel data will go directly from the FPGA to the USB Controller in every cycle, and implementing a small internal RAM as a very small pixel buffer in case there is a temporary stall from the USB controller.

Is that feasible or I will have many frame losses due to transfer bottlenecks?

\$\endgroup\$
  • \$\begingroup\$ What is your data rate from the sensor? How fast can the host process incoming data on the USB connection? \$\endgroup\$ – Elliot Alderson Oct 19 '18 at 14:42
  • \$\begingroup\$ @Elliot Alderson The particular sensor I am working with right now has a maximum data rate 96 MP/s with a clock at 96 MHz but this is intended to be a general design pattern so later other sensors will be used. The host process data pretty fast (CUDA based ISP), but I assume this is not a problem since a frame will be given from the board upon request, so the host is ready to receive. \$\endgroup\$ – Manos Oct 19 '18 at 14:54
  • \$\begingroup\$ Perform Worst-case analysis, starting with calculating throughput requirements to properly size your buffer(s). Think about: Is your sensor a serial or parallel interface? if parallel, how many bits? Are you pushing to the Host or notifying it to retrieve it? How much inter-frame time is there? What is max data rate on the USB interface? You will have to be approx. "sizeOfPixels faster on USB" to keep up with the input, if the input is parallel and has a nominal inter-frame gap. Think about interface latency, arbitration (if any), etc. and then apply the math on a timeline. \$\endgroup\$ – CapnJJ Oct 19 '18 at 15:14
  • \$\begingroup\$ Stall on the USB, as you mentioned, is another WCA thing to consider.... if the aggregate stall time is longer than inter-frame gap it won't matter if you stored everything in a DRAM previously or not. This is another place where the Host specs will come into your WCA timeline calculation... is it a dedicated USB link or not, priority, etc. \$\endgroup\$ – CapnJJ Oct 19 '18 at 15:21
  • \$\begingroup\$ So you need at something like 800 Mb/s data transfer over USB? (Assuming 8 bit pixels.) How many pixels in an image..how long do you need to sustain that data rate to transfer one frame? It doesn't matter how fast the host can process the pixels, how fast can the host transfer data over USB? \$\endgroup\$ – Elliot Alderson Oct 19 '18 at 21:27
1
\$\begingroup\$

The RAM has a buffer functionality. This way you can send data bursts instead of trying to synchronize your incoming and outgoing datastream. Depending on the protocol you might have some significant overhead so you want to have as much data as possible packed in a single frame. Using a RAM buffer you can check if data has been stored and then build your frame and the camera doesn't directly rely on a ready USB connection. So technically you can omit the RAM but I'd at least implement a fifo-buffer (ringbuffer) in the FPGA.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.