Thursday, February 10, 2011

Dense Optical Flow - Polynomial Expansion by Gunner Farneback

It probably requires time to get into details of Polynomial Expansion in order to have a full understanding.

The follow paragraph could be very wrong. Need revisiting later to understand what's under-the-hood.
Farneback method uses Polynomial Expansion to approximate the neighbors of a pixel. The Expansion could be seen as a quadratic equation with Matrices and Vectors as variable and coefficients. This dense optical flow analysis produces a displacement field from two successive video frames. Each displacement vector in the field is estimated by minimizing the 'error' under a 'constraint'. The 'constraint' is an equation A(x)d(x)=delta-b(x) derived from the polynomial expansion. The 'error' is the weighted sum of differences in the pixel neighborhood (x + delta-x) between the images.
Image Pyramids is used to detect large displacements. Also uses Gaussian to smooth out the neighboring displacements. I suppose that's based on the assumption that pixels in proximity move in similar directions.

Sample (fback.cpp)
  • The larger distance movement are not detected with fewer pyramid levels. And there is no noticeable gain in performance.
  • Displacement vectors are drawn at 16 pixels, possibly to avoid cluttering.'
  • Overall is slower than sparse as expected. It seems to have fewer noise. And it detects new objects by nature (Dense).

Two Frame Motion Estimation Based on Polynomial Expansion, Farneback
Polynomial Expansion for Orientation and Motion Estimation, Farneback

No comments:

Post a Comment