0
\$\begingroup\$

I'm very much a beginner in electronics, so I apologize if the concepts I'm trying to understand are very basic. I've been looking into making a simple camera setup with my Arduino and an Adafruit product linked here. From looking at this article, It seems that for a simple setup to work, this would require connecting to a Raspberry Pi at least, due to its requirement for a camera module:

The Raspberry Pi camera board transfers data through an extremely fast camera serial interface (CSI-2) bus directly to the system-on-chip (SoC) processor. It does this through a 15-pin ribbon cable, also known as flexible flat cable (FFC), and connects to the surface mount ZIF 15 socket in the Raspberry Pi board. As you may noted, the camera module on this official Raspberry Pi camera board is identical to the camera modules (ccd imagers) found in many mobile phones.

enter image description here

I'm having trouble understanding this as a concept for a couple reasons.

  1. Why does a camera, which presumably has a set amount of pins that give video output, require a camera module to "process" its data before sending it into the rpi?

    a. How is the raw input of a camera such as in the Adafruit product formatted?

    b. How is the "processed" input of the camera board formatted?

  2. (more dumb question) Assuming there's around 1920x1080 pixels for a video that the camera can take, how is this compressed into the <20 number of pinouts that comes out of the cable?

  3. As I understand it from my limited experience with circuit diagramming, the little markers are inputs and triangles are ground, but what are the rest signifying (do lines like CAM1_DN0 signify inputs?

  4. How could a circuit be designed to read in pixels of a camera and do high-level processing?

Again, apologies if these questions are really basic, but I couldn't seem to find any way of understanding this in a very detailed format online. Any help would be appreciated. Thanks for reading!

\$\endgroup\$
4
  • \$\begingroup\$ Why do you think a camera has a "set amount of pins that give video output"? Video in what format, exactly? \$\endgroup\$
    – Hearth
    Aug 1, 2021 at 5:12
  • \$\begingroup\$ from a basic analysis of, say a JPG format, I would assume it's just color data for each pixel somehow compressed into the pins \$\endgroup\$
    – Daneolog
    Aug 1, 2021 at 5:18
  • \$\begingroup\$ And how exactly do you think that data is transmitted? Even when it comes to the most basic of data transmission protocols, you've got a selection between I²C, SPI, 1-wire, UART, and more--and video transmission is way more high-speed than any of those can handle. \$\endgroup\$
    – Hearth
    Aug 1, 2021 at 5:25
  • \$\begingroup\$ This blog goes through the process of interfacing a bare camera sensor to a PC using an external USB bridge chip. Due to the lower resolution of the camera and that it does not use MIPI they are able to use perf board: circuitvalley.com/2019/12/… \$\endgroup\$ Aug 1, 2021 at 14:33

1 Answer 1

2
\$\begingroup\$

Why does a camera, which presumably has a set amount of pins that give video output, require a camera module to "process" its data before sending it into the rpi?

The camera's output pins may use a different format, but in many cases, the camera module doesn't do any conversion. An example schematic shows the following onboard:

  • Power supply (voltage regulators and decoupling)
  • An oscillator to provide the necessary clock signal to the camera chip.
  • An cryptoprocessor made available on the I2C bus of the board, as well as I2C level shifting.
  • Common-mode chokes for noise suppression on the differential signal lines carrying video from the camera.

There is no intermediate conversion chip between the camera chip and the cable; the common-mode chokes are purely passive components that suppress certain types of noise on the line.

a. How is the raw input of a camera such as in the Adafruit product formatted?

There's no separate "raw input" in the example I found, the output of the camera IC directly conforms to the Camera Serial Interface. For a different module, you'd need to consult the datasheet of the camera IC to confirm the format.

b. How is the "processed" input of the camera board formatted?

It follows the Camera Serial Interface format.

Assuming there's around 1920x1080 pixels for a video that the camera can take, how is this compressed into the <20 number of pinouts that comes out of the cable?

The camera data is sent in a serial format over two differential lanes, with a clock signal alongside. The serial format means that bits of all the pixels in the frame follow one after the other. Per [this reference], "Each lane can run at up to 1Gbit/s (DDR, so the max link frequency is 500MHz)". As far as I can tell based on the abstract of the spec, there are multiple options for the image encoding, including 24-bit RAW.

As I understand it from my limited experience with circuit diagramming, the little markers are inputs and triangles are ground, but what are the rest signifying (do lines like CAM1_DN0 signify inputs?

Those lines are differential signals, they can be considered as outputs of the camera module. The SDA and SCL form an I2C bus, which is used to control and configure the camera, while the CAM1_Dxx pins carry the bulk video data.

How could a circuit be designed to read in pixels of a camera and do high-level processing?

It would need a differential transceiver that can handle the correct IO standard, speed, and encoding. If you're designing a chip you might pay for an IP core to integrate in your chip. If you're using an FPGA you will probably use a transceiver provided onboard the FPGA. An Arduino does not have the appropriate hardware to decode the high-speed video data.

Once you receive and decode the pixel data, you'd place it into a frame buffer and then do interesting stuff to it (e.g. encode it, stream it elsewhere, etc).

Being a member of the MIPI organization (which governs the spec) or otherwise having access to the spec is very helpful.

\$\endgroup\$
3
  • 1
    \$\begingroup\$ I see, thank you very much for your answer. So from what I understand, the reason why there are so little pinouts is because the camera doesn't send all frame data at one point in time; it sends it out on a stream somewhat like a network and it would need a receiver to piece these together. Would it be possible to simply mount a transceiver on a perfboard and wire it to a circuit that processes images? \$\endgroup\$
    – Daneolog
    Aug 1, 2021 at 6:18
  • 1
    \$\begingroup\$ No, as said, the speeds of the signals involved are very high, and you need your board to be designed to carry these kinds of signals. Perfboard can't - and you won't get any of the components you need in a package that you could put on perfboard, either. \$\endgroup\$
    – mmmm
    Aug 1, 2021 at 7:14
  • \$\begingroup\$ @Daneolog mmmm is correct. The signals are too fast. The reason I mention an FPGA is that if it's in your budget, it's probably the best way to attach your own custom processing/logic to an MIPI interface (you'd specifically need an FPGA board with the right transceivers and connector, of course, and it'll be more expensive than the Pi) \$\endgroup\$
    – nanofarad
    Aug 1, 2021 at 15:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.