6
\$\begingroup\$

I have a camera rigidly mounted together with an IMU that consists of 2-dof gyroscopes and 3-dof accelerometers. My aim is to compute the global optical flow of the image captured by the camera, in order to do global motion compensation when compressing the camera video stream. Instead of doing image processing to compute global optical flow, is it possible to obtain the optical flow based purely on the IMU readings?

\$\endgroup\$

closed as off-topic by Chris Stratton, Elliot Alderson, Finbarr, Edgar Brown, Sparky256 Feb 6 at 4:23

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Questions on the use of electronic devices are off-topic as this site is intended specifically for questions on electronics design." – Chris Stratton, Elliot Alderson, Finbarr, Edgar Brown, Sparky256
If this question can be reworded to fit the rules in the help center, please edit the question.

  • \$\begingroup\$ I do not think the first three tags really apply as you are asking if you can do it without the image processing as far as I can tell. \$\endgroup\$ – Kortuk Jan 10 '12 at 0:58
  • \$\begingroup\$ How good is you IMU - they vary vastly. Can you advise barnd and model. What sort of results are you wanting to achieve and what are you able to achieve optically? \$\endgroup\$ – Russell McMahon Jan 10 '12 at 2:20
2
\$\begingroup\$

If by optical flow you mean the movement over time that the board does, it's possible to do that; I implemented a tracking algorithm using a Kalman filter over a STM32 microcontroller to get the orientation of an IMU, but it's a big amount of computation and I'm not sure that it could reduce the load compared to image processing.

Edit:

I actually think it could be feasible, and in fact it's what camera manufacturer do with image stabilization.

Caveats:

  • It's not that hard to do it, but I believe it's quite hard to do it well, or at least in a meaningful way;

  • IMUs are typically oriented to slow-ish and accurate tracking, instead you need to compensate higher frequencies of motion, like vibration and shake. You'll need to tune your tracking accordingly.

In short, you have to convert the x/y/z accelerometer axes and pitch/yaw gyroscope axes to the same set of coordinates, and apply the desired amount of filtering. Then integrate twice the accelerations to get the (linear angular) speed and the velocity of the movement, as desired. That's what you then have to compensate in the video.

You'll probably have to compensate some delays, either in the video or in the sensor readings, to time-match the compensation with the recording.

\$\endgroup\$
  • \$\begingroup\$ More specifically, the instantaneous frame-to-frame optical flow. I do not require the total accumulated movement of the board, which may cause errors to integrate. Any specific details regarding your project? i.e. model of IMU? Doing optical flow on hardware may be more computationally intensive than Kalman filtering. \$\endgroup\$ – Ang Zhi Ping Jan 12 '12 at 2:54
  • \$\begingroup\$ I don't know exactly what you mean with IMU, but if you mean an integrate system that computes automatically the variation of pitch, roll and yaw, I didn't use a standard one. I used a board built at the laboratory where I did that project, it was a normal STM32 microcontroller (don't remember the exact version, i think it was a 32MHz but that's all) with some MEMS sensors (3-axis accelerometer and gyroscope and a magnetometer, but you don't need the latter for your job). We just implemented a basic Kalman filter to merge the data from the three sensors, using quaternions (for gimbal lock)... \$\endgroup\$ – clabacchio Jan 12 '12 at 8:13

Not the answer you're looking for? Browse other questions tagged or ask your own question.