Spontaneous Versus Posed Smiles—Can We Tell the Difference?

  • Bappaditya MandalEmail author
  • Nizar Ouarti
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 460)


Smile is an irrefutable expression that shows the physical state of the mind in both true and deceptive ways. Generally, it shows happy state of the mind, however, ‘smiles’ can be deceptive, for example people can give a smile when they feel happy and sometimes they might also give a smile (in a different way) when they feel pity for others. This work aims to distinguish spontaneous (felt) smile expressions from posed (deliberate) smiles by extracting and analyzing both global (macro) motion of the face and subtle (micro) changes in the facial expression features through both tracking a series of facial fiducial markers as well as using dense optical flow. Specifically the eyes and lips features are captured and used for analysis. It aims to automatically classify all smiles into either ‘spontaneous’ or ‘posed’ categories, by using support vector machines (SVM). Experimental results on large UvA-NEMO smile database show promising results as compared to other relevant methods.


Posed Spontaneous smiles Feature extraction Face analysis 


  1. 1.
    Ambadar, Z., Cohn, J., Reed, L.: All smiles are not created equal: Morphology and timing of smiles perceived as amused, polite, and embarrassed/nervous. Journal of Nonverbal Behavavior 33, 17–34 (2009)CrossRefGoogle Scholar
  2. 2.
    Asthana, A., Zafeiriou, S., Cheng, S., Pantic, M.: Incremental face alignment in the wild. In: CVPR. Columbus, Ohio, USA (2014)Google Scholar
  3. 3.
    Cohn, J., Schmidt, K.: The timing of facial motion in posed and spontaneous smiles. Intl J. Wavelets, Multiresolution and Information Processing 2, 1–12 (2004)Google Scholar
  4. 4.
    Dibeklioglu, H., Valenti, R., Salah, A., Gevers, T.: Eyes do not lie: Spontaneous versus posed smiles. In: ACM Multimedia. pp. 703–706 (2010)Google Scholar
  5. 5.
    Dibeklioglu, H., Salah, A.A., Gevers, T.: Are you really smiling at me? spontaneous versus posed enjoyment smiles. In: IEEE ECCV. pp. 525–538 (2012)Google Scholar
  6. 6.
    Ekman, P.: Telling lies: Cues to deceit in the marketplace, politics, and marriage. WW. Norton & Company, New York (1992)Google Scholar
  7. 7.
    Ekman, P., Hager, J., Friesen, W.: The symmetry of emotional and deliberate facial actions. Psychophysiology 18, 101–106 (1981)CrossRefGoogle Scholar
  8. 8.
    Ekman, P., Rosenberg, E.: What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System. Second ed. Oxford Univ. Press (2005)CrossRefGoogle Scholar
  9. 9.
    Gan, T., Wong, Y., Mandal, B., Chandrasekhar, V., Kankanhalli, M.: Multi-sensor self-quantification of presentations. In: ACM Multimedia. pp. 601–610. Brisbane, Australia (Oct 2015)Google Scholar
  10. 10.
    He, M., Wang, S., Liu, Z., Chen, X.: Analyses of the differences between posed and spontaneous facial expressions. Humaine Association Conference on Affective Computing and Intelligent Interaction pp. 79–84 (2013)Google Scholar
  11. 11.
    Hoque, M., McDuff, D., Picard, R.: Exploring temporal patterns in classifying frustrated and delighted smiles. IEEE Trans. Affective Computing 3, 323–334 (2012)CrossRefGoogle Scholar
  12. 12.
    Hsu, C., Chang, C., Lin, C.: A practical guide to support vector classification (2010)Google Scholar
  13. 13.
    Huijser, M., Gevers, T.: The influence of temporal facial information on the classification of posed and spontaneous enjoyment smiles. Tech. rep., Univ. of Amsterdam (2014)Google Scholar
  14. 14.
    Mandal, B., Ching, S., Li, L., Chandrasekha, V., Tan, C., Lim, J.H.: A wearable face recognition system on google glass for assisting social interactions. In: \(3^{rd}\) International Workshop on Intelligent Mobile and Egocentric Vision, ACCV. pp. 419–433 (Nov 2014)Google Scholar
  15. 15.
    Mandal, B., Eng, H.L.: Regularized discriminant analysis for holistic human activity recognition. IEEE Intelligent Systems 27(1), 21–31 (2012)CrossRefGoogle Scholar
  16. 16.
    Meyer, Y.: Oscillating patterns in image processing and in some nonlinear evolution equations. The Fifteenth Dean Jacquelines B. Lewis Memorial Lectures, American Mathematical Society (2001)zbMATHGoogle Scholar
  17. 17.
    Nguyen, T., Ranganath, S.: Tracking facial features under occlusions and recognizing facial expressions in sign language. In: International Conference on Automatic Face & Gesture Recognition. vol. 6, pp. 1–7 (2008)Google Scholar
  18. 18.
    Ouarti, N., Peyre, G.: Best basis denoising with non-stationary wavelet packets. In: International Conferenc on Image Processing. vol. 6, pp. 3825–3828 (2009)Google Scholar
  19. 19.
    Ouarti, N., SAFRAN, A., LE, B., PINEAU, S.: Method for highlighting at least one moving element in a scene, and portable augmented reality (Aug 22 2013),, wO Patent App. PCT/EP2013/053,216
  20. 20.
    Pfister, T., Li, X., Zhao, G., Pietikainen, M.: Differentiating spontaneous from posed facial expressions within a generic facial expression recognition framework. In: ICCV Workshop. pp. 868–875 (2011)Google Scholar
  21. 21.
    Schmidt, K., Bhattacharya, S., Denlinger, R.: Comparison of deliberate and spontaneous facial movement in smiles and eyebrow raises. Journal of Nonverbal Behavavior 33, 35–45 (2009)CrossRefGoogle Scholar
  22. 22.
    Valstar, M., Pantic, M.: How to distinguish posed from spontaneous smiles using geometric features. In: In Proceedings of ACM ICMI. pp. 38–45 (2007)Google Scholar
  23. 23.
    Yu, X., Han, W., Li, L., Shi, J., Wang, G.: An eye detection and localization system for natural human and robot interaction without face detection. TAROS pp. 54–65 (2011)Google Scholar
  24. 24.
    Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime tv-l1 optical flow. In: In Ann. Symp. German Association Patt. Recogn. pp. 214–223 (2007)Google Scholar
  25. 25.
    Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: Audio, visual, and spontaneous expressions. PAMI 31(1), 39–58 (2009)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Singapore 2017

Authors and Affiliations

  1. 1.Visual Computing DepartmentInstitute for Infocomm ResearchSingaporeSingapore

Personalised recommendations