Study of Facial Micro-expressions in Psychology

Chapter
First Online:

Abstract

The study of micro-expressions has undergone a big change as a result of contemporary development in the areas of human–computer interaction (HCI) and affective computing. This chapter will highlight the need for the study of micro-expressions. It would focus on two major approaches to evolve parameters for the automatic detection of human facial expressions—the facial action coding system (FACS) and facial animation parameters (FAPs). Besides summarizing the major developments in the area of psychology and other areas, it would also explore the ways in which neuropsychological studies can contribute to this domain of knowledge.

Keywords

Micro-expressions Facial expressions Automatic analysis 
This is a preview of subscription content, log in to check access.

References

  1. Aleksic, P. S., & Katsaggelos, A. K. (2006). Automatic facial expression recognition using facial animation parameters and multistream HMMs. IEEE Transaction Information Forensics and Security, 1, 3–11.CrossRefGoogle Scholar
  2. Anderson, K., & McOwen, P. W. (2006). A real-time automated system for recognition of human facial expressions. IEEE Transaction Systems Man and Cybernetics Part B, 36, 96–105.CrossRefGoogle Scholar
  3. Bachorowski, J. A. (1999). Vocal expression and perception of emotion. Current Directions in Psychological Science, 8, 53–57.CrossRefGoogle Scholar
  4. Baron-Cohen, S., Riviere, A., Fukushima, M., French, D., Hadwin, J., Cross, P., et al. (1996). Reading the mind in the face: A cross-cultural and developmental study. Visual Cognition, 3, 39–59.CrossRefGoogle Scholar
  5. Bartlett, M. S., Littlewort, G., Fasel, I., & Movellan, R. (2003). Real time face detection and facial expression recognition: Development and application to human computer interaction. Proceedings of CVPR Workshop on Computer Vision and Pattern Recognition for Human-Computer Interaction. Retrieved from http://mplab.ucsd.edu/~marni/pubs/Bartlett_CVPR_2003.pdf.
  6. Bhushan, B. (2006). A half gateway to the whole: Evidence from face study. Gestalt Theory, 28, 308–315.Google Scholar
  7. Bhushan, B. (2007). Subjective analysis of facial expressions: Inputs from behavioural research for automated systems. Unpublished project report INI-IITK-20060049, Indian Institute of Technology, Kanpur.Google Scholar
  8. Biederman, I. (1987). Recognition-by-components: A theory of human image understanding. Psychological Review, 94, 115–147.PubMedCrossRefGoogle Scholar
  9. Bourel, F., Chibelushi, C. C. & Low, A. A. (2001). Recognition of facial expressions in the presence of occlusion. Proceedings of the Twelfth British Machine Vision Conference, 1, 213–222.Google Scholar
  10. Bradley, M. M., & Lang, P. J. (2000). Measuring emotion: Behavior, feeling, and physiology. In R. D. Lane & L. Nadel (Eds.), Cognitive neuroscience of emotion (pp. 242–276). New York: Oxford University Press.Google Scholar
  11. Bucy, E. P. (2000). Emotional and evaluative consequences of inappropriate leader displays. Communication Research, 27, 194–226.CrossRefGoogle Scholar
  12. Bucy, E. P., & Bradley, S. D. (2004). Presidential expressions and viewer emotion: Counterempathic responses to televised leader displays. Social Sciences Information, 43, 59–94.CrossRefGoogle Scholar
  13. Bullock, M., & Russell, J. (1984). Preschool children’s interpretation of facial expressions of emotion. International Journal of Behavioural Development, 7, 193–214.CrossRefGoogle Scholar
  14. Bullock, M., & Russell, J. (1986). Concepts of emotion in developmental psychology. In C. Izard & P. Read (Eds.), Measuring emotions in infants and children (Vol. 2, pp. 203–237). Cambridge University Press: Cambridge.Google Scholar
  15. Cohn, J. F., Ambadar, Z., & Ekman, P. (2007). Observer-based measurement of facial expression with the Facial Action Coding System. In J. A. Coan & J. B. Allen (Eds.), The handbook of emotion elicitation and assessment, Series in Affective Science. Oxford University Press (pp. 203–221). New York: Oxford.Google Scholar
  16. Darwin, C. (1872). The expression of emotion in man and animals. New York: Oxford University Press.CrossRefGoogle Scholar
  17. De la Torre, F., Yacoob, Y., & Davis, L. (2000). A probabilistic framework for rigid and non-rigid appearance based tracking and recognition. International conference on automatic face and gesture recognition, pp. 491–498.Google Scholar
  18. Ekman, P. (1973). Cross-cultural studies of facial expression. In P. Ekman (Ed.), Darwin and facial expression: A century of research in review (pp. 169–222). New York: Academic Press.Google Scholar
  19. Ekman, P. (1992a). An argument for basic emotions. Cognition and Emotion, 6, 169–200.CrossRefGoogle Scholar
  20. Ekman, P. (1992b). Facial expressions of emotions: an old controversy and new findings. Philosophical Transactions of the Royal Society, B335, 63–69Google Scholar
  21. Ekman, P. (1999). Basic emotions. In T. Dalgleish & M. J. Power (Eds.), Handbook of cognition and emotion (pp. 45–60). New York: Wiley.Google Scholar
  22. Ekman, P. (2001). Telling lies: Clues to deceit in the marketplace, politics and marriage (3rd ed.). New York: W. W. Norton and Company.Google Scholar
  23. Ekman, P. (2003). Emotions revealed (2nd ed.). New York: Times Books.Google Scholar
  24. Ekman, P. (2009). Lie catching and microexpressions: The Philosophy of Deception. Oxford: Oxford University Press.Google Scholar
  25. Ekman, P., & Friesen, W. V. (1974). Nonverbal behavior and psychopathology. In R. J. Friedman & M. Katz (Eds.), The psychology of depression: contemporary theory and research (pp. 3–31). Washington, D. C: Winston and Sons.Google Scholar
  26. Ekman, P., & Friesen, W. V. (1975). Unmasking the face: A guide to recognizing emotions from facial clues. Englewood Cliffs: Prentice-Hall.Google Scholar
  27. Ekman, P., & Friesen, W. V. (1978). Manual for the facial action coding system. San Francisco: Consulting Psychologists Press.Google Scholar
  28. Ekman, P., & Friesen, W. V. (2003). Unmasking the face. Cambridge, MA: Malor Books.Google Scholar
  29. Ekman, P., Friesen, W. V., O’Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., et al. (1987). Universals and cultural differences in the judgments of facial expressions of emotion. Journal of Personality and Social Psychology, 53, 712–717.PubMedCrossRefGoogle Scholar
  30. Endres, J., & Laidlaw, A. (2009). Micro-expression recognition training in medical students: A pilot study. BMC Medical Education, 9, 1–6.CrossRefGoogle Scholar
  31. Essa, I., & Pentland, A. (1995). Facial expression recognition using a dynamic model and motion energy. Proceedings of 5th International conference on computer vision, pp. 360–367.Google Scholar
  32. Esteves, F., & Ohman, A. (1993). Masking the face: Recognition of emotional facial expressions as a function of the parameters of backward masking. Scandinavian Journal of Psychology, 34, 1–18.PubMedCrossRefGoogle Scholar
  33. Farah, M. J., Wilson, K. D., Drain, M., & Tanaka, J. N. (1998). What is ‘special’ about facial perception? Psychological Review, 105, 482–498.PubMedCrossRefGoogle Scholar
  34. Fasel, B., & Luttin, J. (2003). Automatic facial expression analysis: A survey. Pattern Recognition, 36, 259–275.CrossRefGoogle Scholar
  35. Frank, M. G., Herbasz, M., Sinuk, K., Keller, A., & Nolan, C. (2009). I see how you feel: Training laypeople and professionals to recognize fleeting emotions. International Communication Association, Sheraton New York City. Retrieved from http://www.allacademic.com/meta/p15018_index.html.
  36. Gray, J. A. (1994). Three fundamental emotion systems. In P. Ekman & R. J. Davidson (Eds.), The nature of emotion: Fundamental questions (pp. 243–247). New York: Oxford University Press.Google Scholar
  37. Gusnard, D. A., & Raichle, M. E. (2001). Searching for a baseline: Functional imaging and the resting human brain. Nature Reviews Neuroscience, 2, 685−694.Google Scholar
  38. Gray, J. A. (1994). Three fundamental emotion systems. In P. Ekman & R. J. Davidson (Eds.): The nature of emotion: Fundamental questions (pp. 243-247). New York: Oxford University Press. CrossRefGoogle Scholar
  39. Haggard, E. A., & Isaacs, K. S. (1966). Micro-momentary facial expressions as indicators of ego mechanisms in psychotherapy. In L. A. Gottschalk & A. H. Auerbach (Eds.), Methods of research in psychotherapy (pp. 154–165). New York: Appleton-Century-Crofts.Google Scholar
  40. Haidt, J., & Keltner, D. (1999). Culture and facial expression: Open-ended methods find more expressions and a gradient of recognition. Cognition and Emotion, 13, 225–266.CrossRefGoogle Scholar
  41. Hu, C., Chang, Y., Feris, R. & Turk, M. (2004). Manifold based analysis of facial expression. Proceedings of IEEE Workshop on Face Processing in Video, Washington, DC.Google Scholar
  42. Izard, C. E. (1971). The face of emotion. New York: Meridith Corporation.Google Scholar
  43. Izard, C. E. (1991). The psychology of emotions. New York: Plenum Press.CrossRefGoogle Scholar
  44. Kaiser, S., & Wehrle, T. (2001). Facial expressions as indicators of appraisal processes. In K. R. Scherer, A. Schorr, & T. Johnstone (Eds.), Appraisal processes in emotions: Theories, methods, research (pp. 285–300). New York: Oxford University Press.Google Scholar
  45. Kanade, T., Cohn, J., & Tian, Y. (2000). Comprehensive database for facial expression analysis. Proceedings of the IEEE International Conference: Face and Gesture Recognition (AFGR '00), 46–53.Google Scholar
  46. Keil, A., Moratti, S., Sabatinelli, D., Bradley, M. M., & Lang, P. J. (2005). Additive effects of emotional content and spatial selective attention on electrocortical facilitation. Cerebral Cortex, 15, 1187–1197.PubMedCrossRefGoogle Scholar
  47. Kilts, C. D., Egan, G., Gideon, D. A., Ely, T. D., & Hoffman, J. M. (2003). Dissociable neural pathways are involved in the recognition of emotion in static and dynamic facial expressions. Neuroimage, 18, 156–168.PubMedCrossRefGoogle Scholar
  48. Kinsbourne, M., & Bemporad, B. (1984). Lateralization of emotion: A model and the evidence. In N. Fox & R. Davidson (Eds.), The psychobiology of affective development (pp. 259–291). Hillsdale, NJ: Erlbaum.Google Scholar
  49. Kotsia, I., Buciu, I., & Pitas, I. (2008). An analysis of facial expression recognition under partial facial image occlusion. Image and Vision Computing, 26, 1052–1067.CrossRefGoogle Scholar
  50. Kotsia, I., & Pitas, I. (2007). Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Transaction Image Processing, 16, 172–187.CrossRefGoogle Scholar
  51. Lang, P. J. (1995). The emotion probe: Studies of motivation and attention. American Psychologist, 50, 372–385.PubMedCrossRefGoogle Scholar
  52. Larsen, J. T., McGraw, A. P., & Cacioppo, J. T. (2001). Can people feel happy and sad at the same time? Journal of Personality and Social Psychology, 81, 684–696.PubMedCrossRefGoogle Scholar
  53. Larsen, J. T., Norris, C. J., & Cacioppo, J. T. (2003). Effects of positive and negative affect on electromyographic activity over zygomaticus major and corrugator supercilii. Psychophysiology, 40, 776–785.PubMedCrossRefGoogle Scholar
  54. Lazarus, R. S. (1991). Emotion and adaptation. Oxford, UK: Oxford University Press.Google Scholar
  55. Lee, C., & Elgammal, A. (2005). Facial expression analysis using nonlinear decomposable generative models. IEEE International Workshop on Analysis and Modeling of Faces and Gestures, pp. 17–31.Google Scholar
  56. Little, A. C., Burriss, R. P., Jones, B. C., & Roberts, S. C. (2007). Facial appearance affects voting decisions. Evolution and Human Behavior, 28, 18–27.CrossRefGoogle Scholar
  57. Manohar, V., Shreve, M., Goldgof, D., & Sarkar, S. (2008). Finite element modeling of facial deformation in videos for computing strain pattern. 19th International Conference on Pattern Recognition, pp. 1–4.Google Scholar
  58. Manohar, V., Zhang, Y., Goldgof, D., & Sarkar, S. (2007). Facial strain pattern as a soft forensic evidence. Proceedings of the 8 th IEEE workshop on Applications of Computer Vision, p. 42.Google Scholar
  59. Matsumoto, D., & Hwang, H. S. (2011). Judgment of profile views of universal facial expressions of emotion. Emotion, 11, 1223–1229.PubMedCrossRefGoogle Scholar
  60. Matsumoto, D., LeRoux, J. A., Wilson-Cohn, C., Raroque, J., Kooken, K., Ekman, P., et al. (2000). A new test to measure emotion recognition ability: Matsumoto and Ekman’s Japanese and Caucasian Brief Affect Recognition Test (JACBART). Journal of Nonverbal Behavior, 24, 179–209.CrossRefGoogle Scholar
  61. Michalson, L., & Lewis, M. (1985). What do children know about emotions and when do they know it? In M. Lewis & C. Saarni (Eds.), The socialization of emotions (pp. 117–140). London: Plenum Press.CrossRefGoogle Scholar
  62. Milders, M., Sahraie, A., & Logan, S. (2008). Minimum presentation time for masked facial expression discrimination. Cognition and Emotion, 22, 63–82.CrossRefGoogle Scholar
  63. Morris, D. (1977). Manwatching: A field guide to human behavior. New York: Abrams.Google Scholar
  64. Pandzic, I. S., & Forchheimer, R. (2002). MPEG-4 facial animation: The standard, implementation and applications. New Jersey: Wiley.Google Scholar
  65. Panksepp, J. (2007). Neurologizing the psychology of affects: How appraisal-based constructivism and basic emotion theory can coexist. Perspectives on Psychological Science, 2, 281–295.CrossRefGoogle Scholar
  66. Pantic, M., & Patras, I. (2004). Detecting facial actions and their temporal segments in nearly frontal-view face image sequences. Proceedings of IEEE conference on Systems, Man and Cybernetics, 4, 3358–3363.Google Scholar
  67. Pardas, M., & Bonafonte, A. (2002). Facial animation parameters extraction and expression recognition using Hidden Markov Models. Signal Processing: Image Communication, 17, 675–688.Google Scholar
  68. Parrott, W. G. (2000). Emotions in social psychology. Philadelphia: Psychology Press.Google Scholar
  69. Pfister, T., Li, X., Zhao, G., & Pietikäinen, M. (2011). Recognizing spontaneous facial micro-expressions. Proceedings of IEEE International Conference on Computer Vision (ICCV), pp. 1449–1456.Google Scholar
  70. Polikovsky, S., Kameda, Y., & Ohta, Y. (2009). Facial micro-expressions recognition using high speed camera and 3D-gradients descriptor. Conference on imaging for crime detection and prevention, 6.Google Scholar
  71. Porter, S., & Brinke, L. (2008). Reading between the lies: Identifying concealed and falsified emotions in universal facial expressions. Psychological Science, 19, 508–514.PubMedCrossRefGoogle Scholar
  72. Pourtois, G., Grandjean, D., Sander, D., & Vuilleumier, P. (2004). Electrophysiological correlates of rapid spatial orienting towards fearful faces. Cerebral Cortex, 14, 619–633.PubMedCrossRefGoogle Scholar
  73. Raichle, M. E., & Gusnard, D. A. (2005). Intrinsic brain activity sets the stage for expression of motivated behavior. Journal of Comparative Neurology, 493, 167–176.PubMedCrossRefGoogle Scholar
  74. Reuter-Lorenz, P., & Davidson, R. J. (1981). Differential contributions of the two cerebral hemispheres for perception of happy and sad faces. Neuropsychologia, 19, 609–614.PubMedCrossRefGoogle Scholar
  75. Rinn, W. E. (1984). The neuropsychology of facial expression: A review of the neurological and psychological mechanisms for producing facial expressions. Psychological Bulletin, 95, 52–77.PubMedCrossRefGoogle Scholar
  76. Rothwell, J., Bandar, Z., O’shea, J., & Mclean, D. (2006). Silent talker: A new computer-based system for the analysis of facial cues to deception. Applied Cognitive Psychology, 20, 757–777.CrossRefGoogle Scholar
  77. Russell, J. A. (1980). A circumflex model of affect. Journal of Personality and Social Psychology, 39, 1161–1178.CrossRefGoogle Scholar
  78. Scherer, K. R. (1984). On the nature and function of emotion: A component process approach. In K. R. Scherer & P. Ekman (Eds.), Approaches to emotion (pp. 293–318). Hillsdale: Lawrence Erlbaum.Google Scholar
  79. Sebe, N., Lew, M. S., Sun, Y., Cohen, I., Gevers, T., & Huang, T. S. (2007). Authentic facial expression analysis. Image and Vision Computing, 25, 1856–1863.CrossRefGoogle Scholar
  80. Shreve, M., Godavarthy, S., Manohar, V., Goldgof, D., & Sarkar, S. (2009). Towards macro- and micro-expressions\ spotting in video using strain patterns. Applications of Computer Vision (WACV), pp. 1–6.Google Scholar
  81. Silberman, E. K., & Weingartner, H. (1986). Hemispheric lateralization of functions related to emotion. Brain and Cognition, 5, 322–353.PubMedCrossRefGoogle Scholar
  82. Smiley, P., & Huttenlocher, J. (1989). Young children’s acquisition of emotion concepts. In C. Saarni & P. Harris (Eds.), Children’s understanding of emotion (pp. 27–79). Cambridge: Cambridge University Press.Google Scholar
  83. Smith, C. A., & Ellsworth, P. C. (1985). Patterns of cognitive appraisal in emotion. Journal of Personality and Social Psychology, 48, 813–838.PubMedCrossRefGoogle Scholar
  84. Stewart, P. A., Waller, B. M., & Schubert, J. N. (2009). Presidential speechmaking style: Emotional response to micro-expressions of facial affect. Motivation and Emotion, 33, 125–135.CrossRefGoogle Scholar
  85. Tellegen, A., Watson, D., & Clark, L. A. (1999). On the dimensional and hierarchical structure of affect. Psychological Science, 10, 297–303.CrossRefGoogle Scholar
  86. Todorov, A., Mandisodza, A. N., Goren, A., & Hall, C. C. (2005). Inferences of competence from faces predict election outcomes. Science, 308, 1623–1626.PubMedCrossRefGoogle Scholar
  87. Van Gelder, R. S., & Van Gelder, L. (1990). Facial expression and speech: Neuroanatomical considerations. International Journal of Psychology, 25, 141–155.CrossRefGoogle Scholar
  88. Wang, J., & Yin, L. (2007). Static topographic modeling for facial expression recognition and analysis. Computer Vision and Image Understanding, 108, 19–34.CrossRefGoogle Scholar
  89. Warren, G., Schertler, E., & Bull, P. (2009). Detecting deception from emotional and unemotional cues. Journal of Nonverbal Behavior, 33, 59–69.CrossRefGoogle Scholar
  90. Williams, L. M., Liddell, B. J., Rathjen, J., Brown, K. J., Gray, J., Phillips, M., et al. (2004). Mapping the time course of nonconscious and conscious perception of fear: An integration of central and peripheral measures. Human Brain Mapping, 21, 64–74.PubMedCrossRefGoogle Scholar

Copyright information

© Springer India 2015

Authors and Affiliations

  1. 1.Indian Institute of Technology KanpurKanpurIndia

Personalised recommendations