Quantifying Micro-expressions with Constraint Local Model and Local Binary Pattern

  • Wen-Jing Yan
  • Su-Jing Wang
  • Yu-Hsin Chen
  • Guoying Zhao
  • Xiaolan FuEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8925)


Micro-expression may reveal genuine emotions that people try to conceal. However, it’s difficult to measure it. We selected two feature extraction methods to analyze micro-expressions by assessing the dynamic information. The Constraint Local Model (CLM) algorithm is employed to detect faces and track feature points. Based on these points, the ROIs (Regions of Interest) on the face are drawn for further analysis. In addition, Local Binary Pattern (LBP) algorithm is employed to extract texture information from the ROIs and measure the differences between frames. The results from the proposed methods are compared with manual coding. These two proposed methods show good performance, with sensitivity and reliability. This is a pilot study on quantifying micro-expression movement for psychological research purpose. These methods would assist behavior researchers in measuring facial movements on various facets and at a deeper level.


Quantification Micro-expression Dynamic information Constraint Local Model (CLM) Local Binary Pattern (LBP) 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ambadar, Z., Cohn, J.F., Reed, L.I.: All smiles are not created equal: Morphology and timing of smiles perceived as amused, polite, and embarrassed/nervous. Journal of Nonverbal Behavior 33(1), 17–34 (2009)CrossRefGoogle Scholar
  2. 2.
    Asthana, A., Zafeiriou, S., Cheng, S., Pantic, M.: Robust discriminative response map fitting with constrained local models. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3444–3451. IEEE (2013)Google Scholar
  3. 3.
    Bartlett, M., Littlewort, G., Whitehill, J., Vural, E., Wu, T., Lee, K., Eril, A., Cetin, M., Movellan, J.: Insights on spontaneous facial expressions from automatic expression measurement. Dynamic Faces: Insights from Experiments and Computation (2006)Google Scholar
  4. 4.
    Bartlett, M.S., Hager, J.C., Ekman, P., Sejnowski, T.J.: Measuring facial expressions by computer image analysis. Psychophysiology 36(2), 253–263 (1999)CrossRefGoogle Scholar
  5. 5.
    Bartlett, M., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., Movellan, J.: Automatic recognition of facial actions in spontaneous expressions. Journal of Multimedia 1(6), 22–35 (2006)CrossRefGoogle Scholar
  6. 6.
    Cohn, J.F., Sayette, M.A.: Spontaneous facial expression in a small group can be automatically measured: An initial demonstration. Behavior Research Methods 42(4), 1079–1086 (2010)CrossRefGoogle Scholar
  7. 7.
    Cohn, J.F., Zlochower, A.J., Lien, J., Kanade, T.: Automated face analysis by feature point tracking has high concurrent validity with manual facs coding. Psychophysiology 36(1), 35–43 (1999)CrossRefGoogle Scholar
  8. 8.
    Cristinacce, D., Cootes, T.F.: Feature detection and tracking with constrained local models (2006)Google Scholar
  9. 9.
    Ekman, P., Friesen, W.: Nonverbal leakage and clues to deception. Tech. rep., DTIC Document (1969)Google Scholar
  10. 10.
    Ekman, P., Friesen, W., Hager, J.: Facs investigator guide. A human face (2002)Google Scholar
  11. 11.
    Essa, I.A., Pentland, A.P.: Coding, analysis, interpretation, and recognition of facial expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(7), 757–763 (1997)CrossRefGoogle Scholar
  12. 12.
    Frank, M., Ekman, P.: The ability to detect deceit generalizes across different types of high-stake lies. Journal of Personality and Social Psychology 72(6), 1429 (1997)CrossRefGoogle Scholar
  13. 13.
    Frank, M., Maccario, C., Govindaraju, V.: Behavior and security, pp. 86–106. Greenwood Pub. Group, Santa Barbara (2009)Google Scholar
  14. 14.
    Hess, U., Kleck, R.: Differentiating emotion elicited and deliberate emotional facial expressions. European Journal of Social Psychology 20(5), 369–385 (2006)CrossRefGoogle Scholar
  15. 15.
    Hess, U., Kleck, R.E.: The cues decoders use in attempting to differentiate emotionelicited and posed facial expressions. European Journal of Social Psychology 24(3), 367–381 (1994)CrossRefGoogle Scholar
  16. 16.
    Kappas, A., Descteaux, J.: Of butterflies and roaring thunder: nonverbal communication in interaction and regulation of emotion. In: Nonverbal Behavior In Clinical Settings, pp. 45–74 (2003)Google Scholar
  17. 17.
    Koelstra, S., Pantic, M., Patras, I.: A dynamic texture-based approach to recognition of facial actions and their temporal models. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(11), 1940–1954 (2010)CrossRefGoogle Scholar
  18. 18.
    Krumhuber, E., Kappas, A.: Moving smiles: The role of dynamic components for the perception of the genuineness of smiles. Journal of Nonverbal Behavior 29(1), 3–24 (2005)CrossRefGoogle Scholar
  19. 19.
    Krumhuber, E.G., Kappas, A., Manstead, A.S.: Effects of dynamic aspects of facial expressions: a review. Emotion Review 5(1), 41–46 (2013)CrossRefGoogle Scholar
  20. 20.
    Littlewort, G., Whitehill, J., Wu, T., Fasel, I., Frank, M., Movellan, J., Bartlett, M.: The computer expression recognition toolbox (cert). In: 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), pp. 298–305. IEEE (2011)Google Scholar
  21. 21.
    Ojala, T., Pietikinen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recognition 29(1), 51–59 (1996)CrossRefGoogle Scholar
  22. 22.
    Pantic, M., Patras, I.: Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 36(2), 433–449 (2006)CrossRefGoogle Scholar
  23. 23.
    Pfister, T., Li, X., Zhao, G., Pietikainen, M.: Recognising spontaneous facial micro-expressions. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 1449–1456. IEEE (2011)Google Scholar
  24. 24.
    Porter, S., ten Brinke, L.: Reading between the lies: Identifying concealed and falsified emotions in universal facial expressions. Psychological Science 19(5), 508–514 (2008)CrossRefGoogle Scholar
  25. 25.
    Rahu, M.A., Grap, M.J., Cohn, J.F., Munro, C.L., Lyon, D.E., Sessler, C.N.: Facial expression as an indicator of pain in critically ill intubated adults during endotracheal suctioning. American Journal of Critical Care 22(5), 412–422 (2013)CrossRefGoogle Scholar
  26. 26.
    Sebe, N., Lew, M.S., Sun, Y., Cohen, I., Gevers, T., Huang, T.S.: Authentic facial expression analysis. Image and Vision Computing 25(12), 1856–1863 (2007)CrossRefGoogle Scholar
  27. 27.
    Seidl, U., Lueken, U., Thomann, P.A., Kruse, A., Schrder, J.: Facial expression in Facial expression in alzheimers disease impact of cognitive deficits and neuropsychiatric symptoms disease impact of cognitive deficits and neuropsychiatric symptoms. American Journal of Alzheimer’s Disease and Other Dementias 27(2), 100–106 (2012)CrossRefGoogle Scholar
  28. 28.
    Vrij, A.: Detecting lies and deceit: Pitfalls and opportunities. John Wiley & Sons Ltd., West Sussex (2008)Google Scholar
  29. 29.
    Wu, T., Butko, N.J., Ruvolo, P., Whitehill, J., Bartlett, M.S., Movellan, J.R.: Multilayer architectures for facial action unit recognition. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 42(4), 1027–1038 (2012)CrossRefGoogle Scholar
  30. 30.
    Yan, W.J., Li, X., Wang, S.J., Zhao, G., Liu, Y.J., Chen, Y.H., Fu, X.: Casme: An improved spontaneous micro-expression database and the baseline evaluation. PLoS ONE (2014)Google Scholar
  31. 31.
    Yan, W.J., Wang, S.J., Liu, Y.J., Wu, Q., Fu, X.: For micro-expression recognition: Database and suggestions. Neurocomputing (2014)Google Scholar
  32. 32.
    Yan, W.J., Wu, Q., Liang, J., Chen, Y.H., Fu, X.: How fast are the leaked facial expressions: The duration of micro-expressions. Journal of Nonverbal Behavior, 1–14 (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Wen-Jing Yan
    • 1
    • 2
  • Su-Jing Wang
    • 1
  • Yu-Hsin Chen
    • 1
  • Guoying Zhao
    • 3
  • Xiaolan Fu
    • 1
    Email author
  1. 1.State Key Lab of Brain and Cognitive ScienceInstitute of Psychology, Chinese Academy of SciencesBeijingChina
  2. 2.College of Teacher EducationWenzhou UniversityWenzhouChina
  3. 3.Center for Machine Vision ResearchUniversity of OuluOuluFinland

Personalised recommendations