Face Recognition from Long-Term Observations

  • Gregory Shakhnarovich
  • John W. Fisher
  • Trevor Darrell
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2352)


We address the problem of face recognition from a large set of images obtained over time - a task arising in many surveillance and authentication applications. A set or a sequence of images provides information about the variability in the appearance of the face which can be used for more robust recognition. We discuss different approaches to the use of this information, and show that when cast as a statistical hypothesis testing problem, the classification task leads naturally to an information-theoretic algorithm that classifies sets of images using the relative entropy (Kullback-Leibler divergence) between the estimated density of the input set and that of stored collections of images for each class. We demonstrate the performance of the proposed algorithm on two medium-sized data sets of approximately frontal face images, and describe an application of the method as part of a view-independent recognition system.


Face Recognition Face Image Combination Rule Virtual View Active Appearance Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Peter N. Belhumeur, Joao P. Hespanha, and David J. Kriegman. Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):711–720, July 1997.Google Scholar
  2. 2.
    Zoran Biuk and Sven Loncaric. Face Recognition from Multi-Pose Image Sequence. In Proceedings of 2nd Int’l Symposium on Image and Signal Processing and Analysis, pages 319–324, Pula, Croatia, 2001.Google Scholar
  3. 3.
    R. Brunelli and T. Poggio. Face recognition: features vs. templates. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(10):1042–1052, 1993.CrossRefGoogle Scholar
  4. 4.
    Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. Wiley and Sons, New York, 1991.zbMATHGoogle Scholar
  5. 5.
    Trevor Darrell, David Demirdjian, Neal Checka, and Pedro Felzenswalb. Plan-view trajectory estimation with dense stereo background models. In Proceedings of the International Conference on Computer Vision, Vancouver, BC, July 2001.Google Scholar
  6. 6.
    G.J. Edwards, C.J. Taylor, and T.F. Cootes. Improving Identification Performance by Integrating Evidence from Sequences. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition, volume 1, pages 486–491, 1999.Google Scholar
  7. 7.
    S. Gong, A. Psarrou, I. Katsoulis, and P. Palavouzis. Tracking and Recognition of Face Sequences. In European Workshop on Combined Real and Synthetic Image Processing for Broadcast and Video Production, pages 96–112, Hamburg, Germany, 1994.Google Scholar
  8. 8.
    Ismail Haritaoglu, David Harwood, and Larry S. Davis. W4: Real-time surveillance of people and their activities. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8), August 2000.Google Scholar
  9. 9.
    Josef Kittler, Mohamad Hatef, Robert P.W. Duin, and Jiri Matas. On Combining Classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(3):226–239, March 1998.Google Scholar
  10. 10.
    Solomon Kullback. Information Theory and Statistics. Wiley and Sons, New York, 1959.zbMATHGoogle Scholar
  11. 11.
    Baback Moghaddam, Tony Jebara, and Alex Pentland. Bayesian face recognition. Pattern Recognition, 33:1771–1782, 2000.CrossRefGoogle Scholar
  12. 12.
    Baback Moghaddam and Alex Pentland. Probabilistic visual learning for object representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):696–710, 1997.CrossRefGoogle Scholar
  13. 13.
    Penio S. Penev and J. J. Atick. Local feature analysis: A general statistical theory for object representation. Network: Computation in Neural Systems, 7(3):477–500, 1996.zbMATHCrossRefGoogle Scholar
  14. 14.
    Louis L. Scharf. Statistical Signal Process: Detection, Estimation, and Time Series Analysis. Addison-Wesley Publishing Company, New York, 1990.Google Scholar
  15. 15.
    Gregory Shakhnarovich, Lily Lee, and Trevor Darrell. Integrated Face and Gait Recognition From Multiple Views. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition, Lihue, HI, December 2001.Google Scholar
  16. 16.
    Paul A. Viola and Michael J. Jones. Robust real-time object detection. Technical report, COMPAQ Cambridge Research Laboratory, Cambridge, MA, February 2001.Google Scholar
  17. 17.
    L. Wiskott, J. M. Fellous, N. Kruger, and C. von der Malsburg. Face recognition by elastic bunch graph matching. In Proceedings of International Conference on Image Processing, pages 456–463, Heidelberg, 1997. Springer-Verlag.Google Scholar
  18. 18.
    Osamu Yamaguchi, Kazuhiro Fukui, and Ken-ichi Maeda. Face recognition using temporal image sequence. In IEEE International Conference on Automatic Face & Gesture Recognition, pages 318–323, Nara, Japan, 1998.Google Scholar
  19. 19.
    Shintaro Yoshizawa and Kunio Tanabe. Dual differential geometry associated with the Kullback-Leibler information on the Gaussian distributions and its 2-parameter deformations. SUT Journal of Mathematics, 35(1):113–137, 1999.zbMATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Gregory Shakhnarovich
    • 1
  • John W. Fisher
    • 1
  • Trevor Darrell
    • 1
  1. 1.Artificial Intelligence LaboratoryMassachusetts Institute of TechnologyUSA

Personalised recommendations