Human Face Identification from Video Based on Frequency Domain Asymmetry Representation Using Hidden Markov Models

  • Sinjini Mitra
  • Marios Savvides
  • B. V. K. Vijaya Kumar
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4105)


In this paper we introduce a novel human face identification scheme from video data based on a frequency domain representation of facial asymmetry. A Hidden Markov Model (HMM) is used to learn the temporal dynamics of the training video sequences of each subject and classification of the test video sequences is performed using the likelihood scores obtained from the HMMs. We apply this method to a video database containing 55 subjects showing extreme expression variations and demonstrate that the HMM-based method performs much better than identification based on the still images using an Individual PCA (IPCA) classifier, achieving more than 30% improvement.


Hide Markov Model Video Sequence Gesture Recognition Facial Asymmetry Automatic Face 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Turk, M., Pentland, A.: Eigenfaces for recognition. Cognitive Neuroscience 3, 71–96 (1991)CrossRefGoogle Scholar
  2. 2.
    Belhumeur, P.N., Hespanha, J.P., Kriegman, D.: Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Transaction on Pattern Analysis and Machine Intelligence (PAMI) 19, 711–720 (1997)CrossRefGoogle Scholar
  3. 3.
    Li, Y.: Dynamic Face Models: Construction and Application. PhD thesis, University of London, Queen Mary (2001)Google Scholar
  4. 4.
    Edwards, G.J., Taylor, C.J., Cootes, T.F.: Improvinf identification performance by integrating evidence from sequences. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 486–491 (1999)Google Scholar
  5. 5.
    Zhou, S., Krueger, V., Chellappa, R.: Face recognition from video: a CONDENSATION approach. In: Proceedings of IEEE Conference on Automatic Face and Gesture Recognition, pp. 221–228 (2002)Google Scholar
  6. 6.
    Borod, J.D., Koff, E., Yecker, S., Santschi, C., Schmidt, J.M.: Facial asymmetry during emotional expression: gender, valence and measurement technique. Psychophysiology 36, 1209–1215 (1998)Google Scholar
  7. 7.
    Martinez, A.M.: Recognizing imprecisely localized, partially occluded and expression variant faces from a single sample per class. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 24, 748–763 (2002)CrossRefGoogle Scholar
  8. 8.
    Troje, N.F., Buelthoff, H.H.: How is bilateral symmetry of human faces used for recognition of novel views? Vision Research 38, 79–89 (1998)CrossRefGoogle Scholar
  9. 9.
    Thornhill, R., Gangstad, S.W.: Facial attractiveness. Transactions in Cognitive Sciences 3, 452–460 (1999)CrossRefGoogle Scholar
  10. 10.
    Mitra, S., Liu, Y.: Local facial asymmetry for expression classification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2004)Google Scholar
  11. 11.
    Liu, Y., Schmidt, K., Cohn, J., Mitra, S.: Facial asymmetry quantification for expression-invariant human identification. Computer Vision and Image Understanding (CVIU) 91, 138–159 (2003)CrossRefGoogle Scholar
  12. 12.
    Mitra, S., Savvides, M., Vijaya Kumar, B.V.K.: Facial asymmetry in the frequency domain - a new robust biometric. In: Kamel, M., Campilho, A.C. (eds.) ICIAR 2005. LNCS, vol. 3656, pp. 1065–1072. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  13. 13.
    Rabiner, L.: A tutorial on Hidden Markov Models and selected applications in speech recognition. Proceedings of IEEE 77, 257–286 (1989)CrossRefGoogle Scholar
  14. 14.
    Kale, A., Rajagopalan, A.N., Cuntoor, N., Krueger, V.: Gait-based recognition of humans using continuous HMMs. In: Proceedings of IEEE Conference on Automatic Face and Gesture Recognition, pp. 336–341 (2002)Google Scholar
  15. 15.
    Lien, J.J.: Automatic recognition of facial expressions using Hidden Markov Models and estimation of expression intensity. Technical Report CMU-RI-TR-98-31, Carnegie Mellon University (1998)Google Scholar
  16. 16.
    Samaria, F., Young, S.: HMM-based architecture for face identification. Image and Vision Computing 12 (1994)Google Scholar
  17. 17.
    Nefian, A.: A Hidden Markov Model-based approach for face detection and recognition. PhD thesis, Georgia Institute of Technology, Atlanta, GA (1999)Google Scholar
  18. 18.
    Liu, X., Chen, T.: Video-based face recognition using adaptive Hidden Markov Models. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Madison, Wisconsin (2003)Google Scholar
  19. 19.
    Kanade, T., Cohn, J.F., Tian, Y.L.: Comprehensive database for facial expression analysis. In: 4th IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–53 (2000)Google Scholar
  20. 20.
    Oppenheim, A.V., Schafer, R.W.: Discrete-time Signal Processing. Prentice Hall, Englewood Cliffs (1989)MATHGoogle Scholar
  21. 21.
    Baum, L., Petrie, T.: Statistical inference for probabilistic functions of finite state Markov chains. Annals of Mathematical Statistics 37, 1554–1563 (1966)MATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Liu, X., Chen, T., Vijaya Kumar, B.V.K.: On modeling variations for face authentication. In: Proceedings of IEEE Conference on Automatic Face and Gesture Recognition, pp. 369–374 (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Sinjini Mitra
    • 1
  • Marios Savvides
    • 2
  • B. V. K. Vijaya Kumar
    • 2
  1. 1.Information Sciences InstituteUniversity of Southern CaliforniaMarina del ReyUSA
  2. 2.Electrical and Computer Engineering DepartmentCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations