Raspberry Pi stereo camera frame synchronization/genlock (video or still image)

Question

I need to frame synchronize a stereo camera for a computer vision application. The synchronization should the best possible, hopefully on a millisecond level. I would like to get about 10 FPS either as video or still images. Resolution requirement is around 2 Mpix.

The application will be mobile stereo road/terrain scanner from a passenger car moving up to say 100 km/h during daytime (thus good lighting conditions). Rolling shutter should not be an issue with Pi camera at FullHD according to this and this video.

My current hardware search has ended up with a Raspberry Pi solution. I think basically I have 2 options with it:

1. Get two standalone Raspberry Pi units each with a camera on it. Synchronize them via an external GPIO trigger. Cost is 2x25+2x16 = £82.

2. Get the new Compute Module Development Kit and connect both cameras to the single board. Couple the two cameras for a simultaneous release with software. Cost is 126+16+16 = £158.

Will both options have similar sync results? Also I would rather like still image output than video output, because I will need to extract given frames from the video anyways. The still images would preferably be uncompressed not to loose image detail, but JPEG is OK too. I am not sure how many FPS can do the GPIO external trigger with still imaging.

Cost is an important factor for me. Using USB webcams probably wont work since USB devices cant be synced easily (see link below). I will be happy with another hardware recommendation if someone knows better stuff within a similar price range.

The origin of this question is on Video Production SE site in a post titled Creating synchronized stereo videos with 2 cameras at low cost. This link can reveal some more background.

Answer 1

I think your both options/ideas (Pi and Compute) can work.

Raspberry Pi GPIO is fast enough (10MHz+), reading GPIO input state every millisecond will be no problem at all.

If you want to process data on Raspberry Pi with OpenCV or something like that - don't expect too much, Pi CPU is not that fast as some people think. Do some experiments with pre-recorded video or images and see how much computing power you have.

If you find that Raspberry is not fast enough - you can use something like this (for image processing):

Nvidia Jetson TK1 Development Kit

- NVIDIA Kepler GPU with 192 CUDA cores
- NVIDIA 4-Plus-1 quad-core ARM Cortex-A15 CPU
- 2 GB memory, 16 GB eMMC
- Gigabit Ethernet, USB 3.0, SD/MMC, miniPCIe
- HDMI 1.4, SATA, Line out/Mic in, RS232 serial port
- Expansion ports for additional display, GPIOs, and high-bandwidth camera interface

I don't know much about this, just read somewhere about these Nvidia products, but it cost less than Pi Compute module (192USD is 115GBP?), and there is a lot more computing power.

Answer 2

After lot of research and even myself trying to set up 2 RPis with one camera each for a frame synchronized stereo capture I did not really come all the way to precisely measure the sync visually but I realized just this at that time from various discussions and tech specs:

Even though it is possible to sync RPi boards themselves via common GPIO signal (for whatever task) it is not possible to sync their cameras for continuous stereo capture. The main reason is that the camera sensors and boards does not support external trigger functionality (neither v1 OmniVision OV5647 nor v2 Sony IMX219PQ camera board). The OV5647 actually has a so-called FREX input pin which could be used for frame sync but it is neither routed out of the sensor to the board nor is it supported in official RPi camera driver/software. The ArduCam board has a FREX pinout on board but still no software to use it.

It may be possible to sync a single shot (no continuous stereo) though - I guess there was a project for a bullet-time photography - I explored their python code at github (cant remember the exact project name now) - they synced RPis via Ethernet and then called raspistill binary synchronously for a single shot. I think this can work even as there was a 1000ms delay at the beginning of raspistill to allow cameras to set exposure (the timout is same on all cameras so the sync should still work). I think different exposure times may be an issue with PiFace though. Once raspistill is started (and hopefully after the first frame) the sensor enters so called "free running mode" where video is streamed from the sensor at given FPS. Because each camera board has its own oscilator the FPS will never be 100% same for two boards even if started at the same time and the frames will drift from each other the longer the video runs.

TL;DR:

While it may be possible to sync a single stereo shot it is not easily possible to frame synchronize two RPi camera boards for continuous stereo capture due to a lack of external trigger feature on the cameras' sensors.