1
\$\begingroup\$

I'm planning to design a depth camera. I'm considering to use the following Camera with the following specs: https://www.waveshare.com/wiki/IMX219-83_Stereo_Camera

Specification
8 Megapixels
Sensor: Sony IMX219
Resolution: 3280 × 2464 (per camera) 

My plan is to integrate also an Audio facilities.

So I searched and looked for a way on how to do that and found the following chip: https://www.cypress.com/documentation/datasheets/cyusb306x-ez-usb-cx3-mipi-csi-2-superspeed-usb-bridge-controller#res579

CYUSB306X: EZ-USB® CX3: MIPI CSI-2 to SuperSpeed USB Bridge Controller

The questions

  1. Is it possible to integrate audio data to the same USB bus from EZ USB CX3 ?
  2. What kind of API should I work with, like for example If I want to integrate opencv, should I use like a command based system from the UART of EZ USB and issue commands from the PC to the Camera module and return back the data on the USB bus ?
  3. Do you suggest other options in terms of design ? easier way ?

There is for sure an interface with Jetson Kit, but I think that's useless, the Jetson only has two CSI Lanes plus it's more expensive, but maybe it's better in terms of a Good CPU and Many bytes of RAM are their. but for custom based solution is not that good ? Plus an overhead of an OS Like Linux which is not needed at all ?

\$\endgroup\$

1 Answer 1

2
\$\begingroup\$

Is it possible to integrate audio data to the same USB bus from EZ USB CX3 ?

yes.

  1. USB hubs, and a separate USB sound card. 10€.
  2. If with "audio" you mean playback: Literally the first page of the CX3 datasheet:

I2S master (transmitter only) at sampling frequencies of 8 kHz, 16 kHz, 32 kHz, 44.1 kHz, 48 kHz, 96 kHz, and 192 kHz

(if in doubt, always read datasheets first)

What kind of API should I work with, like for example If I want to integrate opencv, should I use like a command based system from the UART of EZ USB and issue commands from the PC to the Camera module and return back the data on the USB bus?

Literally first FAQ:

CX3 device firmware follows the UVC standard and binds with the standard UVC driver

So, it's a Video4Linux2 Webcam from the point of view of any application software. Your Linux kernel takes care of all the device handling.

There is for sure an interface with Jetson Kit, but I think that's useless, the Jetson only has two CSI Lanes plus it's more expensive, but maybe it's better in terms of a Good CPU and Many bytes of RAM are their.

I think you'll find that USB3 data rates alone, and image processing on top, require a rather computationally capable platform. I'd strongly recommend you develop on a PC workstation, and then slim down the algorithm until you think it runs on your target embedded platform. (Don't start on the embedded platform. You've got enough to learn as is.)

It's also recommendable that you don't start with "live" video, but with recordings: algorithm development will be much smoother if you can repeat the same input arbitrary many times exactly and only if that works move on to make it more robust against varying input. Same e.g. for machine learning approaches, where you'll need a whole set of recordings to train.

You're right, by modern standards, the original Jetson isn't much of a high-performance computer, but if you're planning to utilize the GPU to do image processing, it will most definitely leave all e.g. RaspberryPis behind. Again, you need to start developing on a desktop machine, understand the requirements of your application, then decide on an embedded device.

\$\endgroup\$
2
  • \$\begingroup\$ Thanks so much for your input. If I decide on using Jetson, what are the options for streaming back data to PC ? Ethernet ? Do you think an OS there like linux is necessary for future development ? My opinion it's not that needed, and the solution by Cybress is really nice, unless I need to use some GPU Work based AI, Right ? plus 1 \$\endgroup\$
    – andreahmed
    Oct 7, 2020 at 10:02
  • 1
    \$\begingroup\$ many new questions, none really electrical engineering. And again, nobody can recommend any platform before you know how much computational power you need, so go and figure that out. \$\endgroup\$ Oct 7, 2020 at 10:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.