In Search of Truth: Analysis of Smile Intensity Dynamics to Detect Deception

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10022)


Detection of deceptive facial expressions, including estimating smile genuineness, is an important and challenging research topic that draws increasing attention from the computer vision and pattern recognition community. The state-of-the-art methods require localizing a number of facial landmarks to extract sophisticated facial characteristics. In this paper, we explore how to exploit fast smile intensity detectors to extract temporal features. This allows for real-time discrimination between posed and spontaneous expressions at the early smile onset phase. We report the results of experimental validation, which indicate high competitiveness of our method for the UvA-NEMO benchmark database.


Affective computing Face analysis Facial expressions Smile genuineness Deception detection Support vector machines 



This work has been supported by the Polish National Science Centre (NCN) under the Grant: DEC-2012/07/B/ST6/01227.


  1. 1.
    Barkhof, E., de Sonneville, L.M., Meijer, C.J., de Haan, L.: Specificity of facial emotion recognition impairments in patients with multi-episode schizophrenia. Schizophr. Res.: Cogn. 2(1), 12–19 (2015)Google Scholar
  2. 2.
    Ross, E.D., Pulusu, V.K.: Posed versus spontaneous facial expressions are modulated by opposite cerebral hemispheres. Cortex 49(5), 1280–1291 (2013)CrossRefGoogle Scholar
  3. 3.
    Trutoiu, L.C., Carter, E.J., Pollard, N., Cohn, J.F., Hodgins, J.K.: Spatial and temporal linearities in posed and spontaneous smiles. ACM Trans. Appl. Percept. 11(3), 12:1–12:15 (2014)CrossRefGoogle Scholar
  4. 4.
    Nalepa, J., Kawulok, M.: Adaptive memetic algorithm enhanced with data geometry analysis to select training data for SVMs. Neurocomputing 185, 113–132 (2016)CrossRefGoogle Scholar
  5. 5.
    Dibeklioğlu, H., Salah, A.A., Gevers, T.: Recognition of genuine smiles. IEEE Trans. Multimedia 17(3), 279–294 (2015)CrossRefGoogle Scholar
  6. 6.
    Wu, P., Liu, H., Zhang, X.: Spontaneous versus posed smile recognition using discriminative local spatial-temporal descriptors. In: Proceedings of the IEEE ICASSP, pp. 1240–1244 (2014)Google Scholar
  7. 7.
    Calder, A.J., Young, A.W.: Understanding the recognition of facial identity and facial expression. Nature Rev.: Neurosci. 6, 641–651 (2005)CrossRefGoogle Scholar
  8. 8.
    Martinez, B., Valstar, M.F.: Advances, challenges, and opportunities in automatic facial expression recognition. In: Kawulok, M., Celebi, M.E., Smolka, B. (eds.) Advances in Face Detection and Facial Image Analysis, pp. 63–100. Springer, Heidelberg (2016)CrossRefGoogle Scholar
  9. 9.
    Krumhuber, E.G., Likowski, K.U., Weyers, P.: Facial mimicry of spontaneous and deliberate Duchenne and non-Duchenne smiles. J. Nonverbal Behav. 38(1), 1–11 (2014)CrossRefGoogle Scholar
  10. 10.
    Girard, J.M., Cohn, J.F., Jeni, L.A., Sayette, M.A., De la Torre, F.: Spontaneous facial expression in unscripted social interactions can be measured automatically. Behav. Res. Methods 47(4), 1136–1147 (2015)CrossRefGoogle Scholar
  11. 11.
    Valstar, M.F., Pantic, M., Ambadar, Z., Cohn, J.F.: Spontaneous vs. posed facial behavior: automatic analysis of brow actions. In: Proceedings of the ICMI, pp. 162–170 (2006)Google Scholar
  12. 12.
    Cohn, J.F., Schmidt, K.L.: The timing of facial motion in posed and spontaneous smiles. Int. J. Wavelets Multiresolut. Inf. Process. 02(02), 121–132 (2004)CrossRefGoogle Scholar
  13. 13.
    Sénéchal, T., Turcot, J., el Kaliouby, R.: Smile or smirk? Automatic detection of spontaneous asymmetric smiles to understand viewer experience. In: Proceedings of the IEEE FG, pp. 1–8 (2013)Google Scholar
  14. 14.
    Dibeklioğlu, H., Valenti, R., Salah, A.A., Gevers, T.: Eyes do not lie: spontaneous versus posed smiles. In: ACM International Conference on Multimedia, pp. 1–4 (2010)Google Scholar
  15. 15.
    Gan, Q., Wu, C., Wang, S., Ji, Q.: Posed and spontaneous facial expression differentiation using deep boltzmann machines. In: 2015 International Conference on Affective Computing and Intelligent Interaction (ACII), pp. 643–648. IEEE (2015)Google Scholar
  16. 16.
    Zhao, G., Pietikäinen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007)CrossRefGoogle Scholar
  17. 17.
    Smolka, B., Nurzynska, K.: Power LBP: a novel texture operator for smiling and neutral facial display classification. Procedia Comp. Sci. 51, 1555–1564 (2015)CrossRefGoogle Scholar
  18. 18.
    Abouelenien, M., Pérez-Rosas, V., Mihalcea, R., Burzo, M.: Deception detection using a multimodal approach. In: Proceedings of the ICMI, pp. 58–65. ACM, New York (2014)Google Scholar
  19. 19.
    Cohn, J., Reed, L., Moriyama, T., Xiao, J., Schmidt, K., Ambadar, Z.: Multimodal coordination of facial action, head rotation, and eye motion during spontaneous smiles. In: Proceedings of the IEEE FG, pp. 129–138 (2004)Google Scholar
  20. 20.
    Valstar, M.F., Gunes, H., Pantic, M.: How to distinguish posed from spontaneous smiles using geometric features. In: Proceedings of the ICMI, pp. 38–45. ACM (2007)Google Scholar
  21. 21.
    Rajoub, B.A., Zwiggelaar, R.: Thermal facial analysis for deception detection. IEEE Trans. Inf. Forensics Secur. 9(6), 1015–1023 (2014)CrossRefGoogle Scholar
  22. 22.
    Kawulok, M., Szymanek, J.: Precise multi-level face detector for advanced analysis of facial images. IET Image Process. 6(2), 95–103 (2012)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Kawulok, M., Nalepa, J.: Towards robust SVM training from weakly labeled large data sets. In: Proceedings of the ACPR, pp. 464–468 (2015)Google Scholar
  24. 24.
    Dibeklioğlu, H., Salah, A.A., Gevers, T.: Are you really smiling at me? Spontaneous versus posed enjoyment smiles. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part III. LNCS, vol. 7574, pp. 525–538. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  25. 25.
    Pfister, T., Li, X., Zhao, G., Pietikäinen, M.: Differentiating spontaneous from posed facial expressions within a generic facial expression recognition framework. In: Proceedings of the IEEE ICCV, pp. 868–875 (2011)Google Scholar
  26. 26.
    Kawulok, M., Wu, J., Hancock, E.R.: Supervised relevance maps for increasing the distinctiveness of facial images. Pattern Recogn. 44(4), 929–939 (2011)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Silesian University of TechnologyGliwicePoland

Personalised recommendations