Skip to main content

Investigating the Dynamics of Facial Expression

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNIP,volume 4292)

Abstract

This paper is concerned with capturing the dynamics of facial expression. The dynamics of facial expression can be described as the intensity and timing of a facial expression and its formation. To achieve this we developed a technique that can accurately classify and differentiate between subtle and similar expressions, involving the lower face. This is achieved by using Local Linear Embedding (LLE) to reduce the dimensionality of the dataset and applying Support Vector Machines (SVMs) to classify expressions. We then extended this technique to estimate the dynamics of facial expression formation in terms of intensity and timing.

Keywords

  • Support Vector Machine
  • Facial Expression
  • Facial Expression Recognition
  • Facial Action
  • Linear Embedding

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/11919629_35
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   149.00
Price excludes VAT (USA)
  • ISBN: 978-3-540-48627-5
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   189.00
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Darwin, C., Ekman, P.: The expression of the emotions in man and animal. The University of Chicago Press, Chicago (1998); 1st edition in 1872, 2nd edition in 1889, 3rd edition with additional commentry by Ekman, P. (1998)

    Google Scholar 

  2. Cohn, J., Kanade, T., Moriyama, T., Ambadar, Z., Xiao, J., Gao, J., Imamura, H.: A comparative study of alternative facs coding algorithms. Technical Report CMU-RI-TR-02-06, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA (2001)

    Google Scholar 

  3. Ekman, P., Friesen, W., Hager, J.: Facial action coding system. Consulting Psychologists Press, Palo Alto (1978)

    Google Scholar 

  4. Cohn, J.F., Schmidt, K., Gross, R., Ekman, P.: Individual differences in facial expression: Stability over time, relation to self-reported emotion, and ability to inform person identification. In: Proceedings of Intel. Conf. On Multimedia and Expo., 2001 (2002)

    Google Scholar 

  5. Ambadar, Z., Schooler, J., Cohn, J.: Deciphering the enigmatic face: The importance of facial dynamics to interpreting subtle facial expressions. Psychological Science (2005)

    Google Scholar 

  6. Bartlett, M.S., Movellan, J., Littlewort, G., Braathen, B., Frank, M.G., Sejnowski, T.J.: Towards automatic recognition of spontaneous facial actions. In: Ekman, P. (ed.) What the face reveals. Oxford University Press, Oxford (2003)

    Google Scholar 

  7. Cohen, I., Sebe, N., Huang, L.C.A.G.: Facial expression recognition from video sequences: Temporal and static modeling. Computer Vision and Image Understanding, 91(1-2) (2003)

    Google Scholar 

  8. Ekman, P.: Facial expression of emotion: New findings, new questions (1992)

    Google Scholar 

  9. Hadid, A., Pietikdinen, M.: An experimental investigation about the integration of facial dynamics in video-based face recognition. ELCVIA 5, 1–13 (2005)

    Google Scholar 

  10. Littlewort, G., Bartlett, M.S., Fasel, I., Susskind, J., Movellan, J.: Dynamics of facial expression extracted automatically from video. Computer vision and Image understanding (2006)

    Google Scholar 

  11. Ekman, P., Friesen, W., Hager, J.: Facial Action Coding System Manual (2002)

    Google Scholar 

  12. Ghent, J., McDonald, J., Harper, J.: Statistical model for expression generation using the facial action coding system. Technical Report NUIM-CS-TR-2003-01, NUI Maynooth (2003)

    Google Scholar 

  13. Saul, L.K., Roweis, S.T.: An introduction to locally linear embedding (2001), http://www.cs.toronto.edu/roweis/lle/publications.html

  14. Schmidt, K., Cohn, J.: Dynamics of facial expression: Normative characteristics and individual differences. In: IEEE International Conference on Multimedia and Expo (ICME 2001), pp. 728–731 (2001)

    Google Scholar 

  15. Cohn, J.: Automated analysis of the configuration and timing of facial expression. In: Ekman, P., Rosenberg, E. (eds.) Afterword of What the face reveals (2nd edn.): Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS) (2005)

    Google Scholar 

  16. Pantic, M., Patras, I.: Dynamics of facial expression: Recognition of facial actions and their temporal segments from face profile image sequences. SMC-B 36, 433–449 (2006)

    Google Scholar 

  17. Zhang, Y., Ji, Q.: Facial expression understanding in image sequences using dynamic and active visual information fusion. In: ICCV, pp. 113–118 (2003)

    Google Scholar 

  18. Donato, G., Bartlett, M., Hager, J., Ekman, P., Sejnowski, T.: Classifying facial actions. IEEE Transactions on Pattern Analysis and Machine Intelligence 21(10), 974–989 (1999)

    CrossRef  Google Scholar 

  19. Bartlett, M.S., Littlewort, G., Frank, M., lainscsek, C., Fasel, I., Movellan, J.: Recognising facial expression: Machine learning and application to spontaneous behaviour. In: IEEE International conference on computer vision and pattern recognition (2005)

    Google Scholar 

  20. Bartlett, M.S., Littlewort, G., Lainscsek, C., Fasel, I., Movellan, J.: Machine learning methods for fully automatic recognition of facial expressions and facial actions. In: IEEE International conference on systems, man and cybernetics, pp. 592–597 (2004)

    Google Scholar 

  21. Bartlett, M.S., Braathen, B., Movellan, J.: Automatic analysis of spontaneous facial behavior: A final report. MPLAB-TR-2001-06, Institute for Neural Computation, University of California, San Diego (2001)

    Google Scholar 

  22. Bartlett, M.S., Littlewort, G., Fasel, I., Movellan, J.: Real-time face detection and facial expression recognition: Development and applications to human computer interaction (2003)

    Google Scholar 

  23. Bartlett, M.S., Littlewort, G., Braathen, B., Sejnowski, T., Movellan, J.: A prototype for automatic recognition of spontaneous facial actions (2003)

    Google Scholar 

  24. Bartlet, M., Littlewort, G., Lainscsek, C., Fasel, I., Frank, M., Movellan, J.: Fully automatic facial action recognition in spontaneous behavior bartlett (2006)

    Google Scholar 

  25. Goneid, A., el Kaliouby, R.: Facial feature analysis of spontaneous facial expression. In: Proceedings of the 10th International AI Applications Conference (2002)

    Google Scholar 

  26. Fasel, B., Luettin, J.: Automatic facial expression analysis: A survey. Pattern Recognition 36(1), 259–275 (2003)

    MATH  CrossRef  Google Scholar 

  27. Saul, L.K., Roweis, S.T.: Think globally, fit locally: unsupervised learning of low dimensional manifolds. Journal of Machine Learning Research 4 (2003)

    Google Scholar 

  28. Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by local linear embedding. Science 290, 2323–2326 (2000)

    CrossRef  Google Scholar 

  29. Chang, Y., Yeung, D.Y.: Robust local linear embedding. Technical report, Department of Computer Science, Hong Kong University of Science and Technology, HKUST-CS05-12 (2005)

    Google Scholar 

  30. Hadid, A., PietikAanien, M.: Efficient locally linear embeddings of imperfect manifolds. In: Proceedings of the Third International Conference on Machine Learning and Data Mining in Pattern Recognition, Leipzig, Germany, pp. 188–201 (2003)

    Google Scholar 

  31. Ridder, D.D., Kouropteva, O., Okun, O., PietikAanien, M., Duin, R.P.W.: Supervised locally linear embedding. In: ICANN 2003, pp. 333–341 (2003)

    Google Scholar 

  32. Kouropteva, O., Okun, O., Hadid, A., Soriano, M., Marcos, S., PietikAainen, M.: Beyond locally linear embedding algorithm. Technical report, Department of Electrical and Information Engerring, University of Oulu, Oulu, Finland, MVG-01-2002 (2002)

    Google Scholar 

  33. Ridder, D.D., Duin, R.P.W.: Locally linear embedding for classification. Technical report, Pattern Recognition Group, Department of Imaging Science and Technology, Delft University of Technology, Delft, The Netherlands, PH-2002-01 (2002)

    Google Scholar 

  34. Campbell, C.: kernel methods: A survey of current techniques. Neurocomputing 48, 63–84 (2002)

    MATH  CrossRef  Google Scholar 

  35. Rogers, S., Williams, R.D., Campbell, C.: Class prediction with Microarray Datasets. In: BioInformatics with computational intelligence paradigms. Springer, Heidelberg (2004)

    Google Scholar 

  36. Rogers, S.: Machine Learning Techniques for Microarray Analysis. Faculty of engineering mathematics, University of Bristol (2004)

    Google Scholar 

  37. ScholKopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2002)

    Google Scholar 

  38. Ghent, J.: A Computational Model of Facial Expression. PhD thesis, National University of Ireland Maynooth, Co., kildare, Ireland (2005)

    Google Scholar 

  39. Cohn, J., Kanade: Cohn-kanade au-coded facial expression database. Technical report, Pittsburgh University (1999)

    Google Scholar 

  40. Gower, J.C.: Generalised procrustes analysis. Psychometrika 40, 33–50 (1975)

    MATH  CrossRef  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Reilly, J., Ghent, J., McDonald, J. (2006). Investigating the Dynamics of Facial Expression. In: , et al. Advances in Visual Computing. ISVC 2006. Lecture Notes in Computer Science, vol 4292. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11919629_35

Download citation

  • DOI: https://doi.org/10.1007/11919629_35

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-48626-8

  • Online ISBN: 978-3-540-48627-5

  • eBook Packages: Computer ScienceComputer Science (R0)