International Journal of Computer Vision

, Volume 56, Issue 3, pp 221–255 | Cite as

Lucas-Kanade 20 Years On: A Unifying Framework

  • Simon Baker
  • Iain Matthews
Article

Abstract

Since the Lucas-Kanade algorithm was proposed in 1981 image alignment has become one of the most widely used techniques in computer vision. Applications range from optical flow and tracking to layered motion, mosaic construction, and face coding. Numerous algorithms have been proposed and a wide variety of extensions have been made to the original formulation. We present an overview of image alignment, describing most of the algorithms and their extensions in a consistent framework. We concentrate on the inverse compositional algorithm, an efficient algorithm that we recently proposed. We examine which of the extensions to Lucas-Kanade can be used with the inverse compositional algorithm without any significant loss of efficiency, and which cannot. In this paper, Part 1 in a series of papers, we cover the quantity approximated, the warp update rule, and the gradient descent approximation. In future papers, we will cover the choice of the error function, how to allow linear appearance variation, and how to impose priors on the parameters.

image alignment Lucas-Kanade a unifying framework additive vs. compositional algorithms forwards vs. inverse algorithms the inverse compositional algorithm efficiency steepest descent Gauss-Newton Newton Levenberg-Marquardt 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Baker, S. and Kanade, T. 2000. Limits on super-resolution and how to break them. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 372–379.Google Scholar
  2. Baker, S. and Matthews, I. 2001. Equivalence and efficiency of image alignment algorithms. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 1090–1097.Google Scholar
  3. Baker, S. and Matthews, I. 2002. Lucas-kande 20 years on: A unifying framework: Part 1. Technical Report CMU-RI-TR-02-16, Carnegie Mellon University Robotics Institute.Google Scholar
  4. Bergen, J.R., Anandan, P., Hanna, K.J., and Hingorani, R. 1992. Hierarchical model-based motion estimation. In Proceedings of the European Conference on Computer Vision, pp. 237–252.Google Scholar
  5. Black, M. and Jepson, A. 1998. Eigen-tracking: Robust matching and tracking of articulated objects using a view-based representation. International Journal of Computer Vision, 36(2):101–130.Google Scholar
  6. LaCascia, M., Sclaroff, S., and Athitsos, V. 2000. Fast, reliable head tracking under varying illumination: An approach based on registration of texture-mapped 3D models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(6):322–336.Google Scholar
  7. Christensen, G.E. and Johnson, H.J. 2001. Image consistent registration. IEEE Transactions on Medical Imaging, 20(7):568– 582.Google Scholar
  8. Cootes, T.F., Edwards, G.J., and Taylor, C.J. 1998. Active appearance models. In Proceedings of the European Conference on Computer Vision, vol. 2, pp. 484–498.Google Scholar
  9. Dellaert, F. and Collins, R. 1999. Fast image-based tracking by selective pixel integration. In Proceedings of the ICCVWorkshop on Frame-Rate Vision, pp. 1–22.Google Scholar
  10. Gill, P.E., Murray, W., and Wright, M.H. 1986. Practical Optimization. Academic Press.Google Scholar
  11. Gleicher, M. 1997. Projective registration with difference decomposition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 331–337.Google Scholar
  12. Hager, G.D. and Belhumeur, P.N. 1998. Efficient region tracking with parametric models of geometry and illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(10):1025– 1039.Google Scholar
  13. Lucas, B. and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Proceedings of the International Joint Conference on Artificial Intelligence, pp. 674– 679.Google Scholar
  14. Press, W.H., Flannery, B.P., Teukolsky, S.A., and Vetterling, W.T. 1992. Numerical Recipes in C: The Art of Scientific Computing, 2nd edn., Cambridge University Press.Google Scholar
  15. Sclaroff, S. and Isidoro, J. 1998. Active blobs. In Proceedings of the 6th IEEE International Conference on Computer Vision, pp. 1146–1153.Google Scholar
  16. Shum, H.-Y. and Szeliski, R. 2000. Construction of panoramic image mosaics with global and local alignment. International Journal of Computer Vision, 16(1):63–84.Google Scholar
  17. Szeliski, R. and Golland, P. 1998. Stereo matching with transparency and matting. In Proceedings of the 6th IEEE International Conference on Computer Vision, pp. 517– 524.Google Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Simon Baker
    • 1
  • Iain Matthews
    • 1
  1. 1.The Robotics InstituteCarnegie Mellon UniversityUSA

Personalised recommendations