Lucas-Kanade 20 Years On: A Unifying Framework

Abstract

Since the Lucas-Kanade algorithm was proposed in 1981 image alignment has become one of the most widely used techniques in computer vision. Applications range from optical flow and tracking to layered motion, mosaic construction, and face coding. Numerous algorithms have been proposed and a wide variety of extensions have been made to the original formulation. We present an overview of image alignment, describing most of the algorithms and their extensions in a consistent framework. We concentrate on the inverse compositional algorithm, an efficient algorithm that we recently proposed. We examine which of the extensions to Lucas-Kanade can be used with the inverse compositional algorithm without any significant loss of efficiency, and which cannot. In this paper, Part 1 in a series of papers, we cover the quantity approximated, the warp update rule, and the gradient descent approximation. In future papers, we will cover the choice of the error function, how to allow linear appearance variation, and how to impose priors on the parameters.

This is a preview of subscription content, access via your institution.

References

  1. Baker, S. and Kanade, T. 2000. Limits on super-resolution and how to break them. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 372–379.

  2. Baker, S. and Matthews, I. 2001. Equivalence and efficiency of image alignment algorithms. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 1090–1097.

    Google Scholar 

  3. Baker, S. and Matthews, I. 2002. Lucas-kande 20 years on: A unifying framework: Part 1. Technical Report CMU-RI-TR-02-16, Carnegie Mellon University Robotics Institute.

  4. Bergen, J.R., Anandan, P., Hanna, K.J., and Hingorani, R. 1992. Hierarchical model-based motion estimation. In Proceedings of the European Conference on Computer Vision, pp. 237–252.

  5. Black, M. and Jepson, A. 1998. Eigen-tracking: Robust matching and tracking of articulated objects using a view-based representation. International Journal of Computer Vision, 36(2):101–130.

    Google Scholar 

  6. LaCascia, M., Sclaroff, S., and Athitsos, V. 2000. Fast, reliable head tracking under varying illumination: An approach based on registration of texture-mapped 3D models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(6):322–336.

    Google Scholar 

  7. Christensen, G.E. and Johnson, H.J. 2001. Image consistent registration. IEEE Transactions on Medical Imaging, 20(7):568– 582.

    Google Scholar 

  8. Cootes, T.F., Edwards, G.J., and Taylor, C.J. 1998. Active appearance models. In Proceedings of the European Conference on Computer Vision, vol. 2, pp. 484–498.

    Google Scholar 

  9. Dellaert, F. and Collins, R. 1999. Fast image-based tracking by selective pixel integration. In Proceedings of the ICCVWorkshop on Frame-Rate Vision, pp. 1–22.

  10. Gill, P.E., Murray, W., and Wright, M.H. 1986. Practical Optimization. Academic Press.

  11. Gleicher, M. 1997. Projective registration with difference decomposition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 331–337.

  12. Hager, G.D. and Belhumeur, P.N. 1998. Efficient region tracking with parametric models of geometry and illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(10):1025– 1039.

    Google Scholar 

  13. Lucas, B. and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Proceedings of the International Joint Conference on Artificial Intelligence, pp. 674– 679.

  14. Press, W.H., Flannery, B.P., Teukolsky, S.A., and Vetterling, W.T. 1992. Numerical Recipes in C: The Art of Scientific Computing, 2nd edn., Cambridge University Press.

  15. Sclaroff, S. and Isidoro, J. 1998. Active blobs. In Proceedings of the 6th IEEE International Conference on Computer Vision, pp. 1146–1153.

  16. Shum, H.-Y. and Szeliski, R. 2000. Construction of panoramic image mosaics with global and local alignment. International Journal of Computer Vision, 16(1):63–84.

    Google Scholar 

  17. Szeliski, R. and Golland, P. 1998. Stereo matching with transparency and matting. In Proceedings of the 6th IEEE International Conference on Computer Vision, pp. 517– 524.

Download references

Author information

Affiliations

Authors

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Baker, S., Matthews, I. Lucas-Kanade 20 Years On: A Unifying Framework. International Journal of Computer Vision 56, 221–255 (2004). https://doi.org/10.1023/B:VISI.0000011205.11775.fd

Download citation

  • image alignment
  • Lucas-Kanade
  • a unifying framework
  • additive vs. compositional algorithms
  • forwards vs. inverse algorithms
  • the inverse compositional algorithm
  • efficiency
  • steepest descent
  • Gauss-Newton
  • Newton
  • Levenberg-Marquardt