Skip to main content

Effects of Dynamic Attributes of Smiles in Human and Synthetic Faces: A Simulated Job Interview Setting

Abstract

We examined the effects of the temporal quality of smile displays on impressions and decisions made in a simulated job interview. We also investigated whether similar judgments were made in response to synthetic (Study 1) and human facial stimuli (Study 2). Participants viewed short video excerpts of female interviewees exhibiting dynamic authentic smiles, dynamic fake smiles, or neutral expressions, and rated them with respect to a number of attributes. In both studies, perceivers’ judgments and employment decisions were significantly shaped by the temporal quality of smiles, with dynamic authentic smiles generally leading to more favorable job, person, and expression ratings than dynamic fake smiles or neutral expressions. Furthermore, authentically smiling interviewees were judged to be more suitable and were more likely to be short-listed and selected for the job. The findings show a high degree of correspondence in the effects created by synthetic and human facial stimuli, suggesting that temporal features of smiles similarly influence perceivers’ judgments and decisions across the two types of stimulus.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

Notes

  1. 1.

    In the context of this paper, the temporal form or quality of smiles refers to the dynamic or moving aspect of facial expressions and is operationalized in terms of its onset, apex, and offset duration.

  2. 2.

    Clearly, the Duchenne marker as a morphological feature is a perceptible signal in social interaction separate from the effect of temporal features. However, we argue that temporal dynamics may themselves be sufficient to shape perceptions and strategic decisions independent of this morphological marker (see Krumhuber et al. 2007a for a similar approach).

References

  1. Ambadar, Z., Schooler, J., & Cohn, J. (2005). Deciphering the enigmatic face: The importance of facial dynamics in interpreting subtle facial expressions. Psychological Science, 16, 403–410.

    PubMed  Article  Google Scholar 

  2. Ambady, N., Bernieri, F., & Richeson, J. (2000). Toward a histology of social behavior: Judgmental accuracy from thin slices of the behavioral stream. In M. Zanna (Ed.), Advances in experimental social psychology (Vol. 32, pp. 201–271). Boston: Academic Press.

    Google Scholar 

  3. Ambady, N., & Rosenthal, R. (1992). Thin slices of behavior as predictors of interpersonal consequences: A meta-analysis. Psychological Bulletin, 2, 256–274.

    Article  Google Scholar 

  4. Ambady, N., & Rosenthal, R. (1993). Half a minute: Predicting teacher evaluations from thin slices of behavior and physical attractiveness. Journal of Personality and Social Psychology, 64, 431–441.

    Article  Google Scholar 

  5. Bailenson, J. N., Blascovich, J., Beall, A. C., & Loomis, J. M. (2001). Equilibrium revisited: Mutual gaze and personal space in virtual environments. Presence, 10, 583–598.

    Article  Google Scholar 

  6. Bassili, J. N. (1978). Facial motion in the perception of faces and of emotional expression. Journal of Experimental Psychology: Human Perception and Performance, 4, 373–379.

    PubMed  Article  Google Scholar 

  7. Bassili, J. N. (1979). Emotion recognition: The role of facial movement and the relative importance of upper and lower areas of the face. Journal of Personality and Social Psychology, 37, 2049–2058.

    PubMed  Article  Google Scholar 

  8. Bente, G., Krämer, N. C., Petersen, A., & de Ruiter, J. (2001). Computer animated movement and person perception: Methodological advances in nonverbal behavior research. Journal of Nonverbal Behavior, 25, 151–166.

    Article  Google Scholar 

  9. Blascovich, J. (2001). Social influences within immersive virtual environments. In R. Schroeder (Ed.), The social life of avatars (pp. 127–145). New York: Springer-Verlag.

    Google Scholar 

  10. Blens, H., Krämer, C., & Bente, G. (2003). Virtuelle Verkäufer: Die Wirkung von antromporphen Interface Agenten in WWW und e-commerce. In J. Ziegler, et al. (Eds.), Mensch und Computer 2003. Stuttgart: Teubner.

    Google Scholar 

  11. Bould, E., & Morris, N. (2008). Role of motion signals in recognizing subtle facial expressions of emotion. British Journal of Psychology, 99, 167–189.

    PubMed  Article  Google Scholar 

  12. Bruce, V., & Valentine, T. (1988). When a nod’s as good as a wink: The role of dynamic information in facial recognition. In M. M. Gruneberg, P. E. Morris, & R. N. Sykes (Eds.), Practical aspects of memory: Current research and issues (Vol. 1, pp. 169–174). New York, NY: John Wiley & Sons.

    Google Scholar 

  13. Bugental, D. B. (1986). Unmasking the “polite smile”: Situational and personal determinants of managed affect in adult-child interaction. Personality and Social Psychology Bulletin, 12, 7–16.

    Article  Google Scholar 

  14. BusinessWeek. (2006, March 7). On-the-job video gaming. Retrieved on January 9, 2008. http://www.businessweek.com/magazine/content/06_13/b3977062.htm.

  15. Cosker, D., Paddock, S., Marshall, D., Rosin, P. L., & Rushton, S. (2005). Towards perceptually realistic talking heads: Models, methods and McGurk. ACM Transactions on Applied Perception, 2, 1–18.

    Article  Google Scholar 

  16. Cosker, D., Rosin, P. L., & Marshall, D. (2007). Remapping animation parameters between multiple types of facial model, lecture notes in computer science (LNCS). Proceedings of Mirage, 2007(4418), 365–378.

    Google Scholar 

  17. Dehn, D. M., & van Mulken, S. (2000). The impact of animated interface agents: A review of empirical research. International Journal of Human-Computer Studies, 52, 1–22.

    Article  Google Scholar 

  18. DePaulo, B. (1992). Nonverbal behavior and self-presentation. Psychological Bulletin, 111, 203–243.

    PubMed  Article  Google Scholar 

  19. Edinger, J. A., & Patterson, M. L. (1983). Nonverbal involvement and social control. Psychological Bulletin, 93, 30–56.

    Article  Google Scholar 

  20. Ekman, P. (1985). Telling lies. New York: W. W. Norton.

    Google Scholar 

  21. Ekman, P. (1992). Facial expressions of emotions: New finding, new questions. Psychological Science, 3, 34–38.

    Article  Google Scholar 

  22. Ekman, P., Davidson, R., & Friesen, W. V. (1990). The Duchenne smile: Emotional expression and brain physiology II. Journal of Personality and Social Psychology, 58, 342–353.

    PubMed  Article  Google Scholar 

  23. Ekman, P., & Friesen, W. V. (1978). The facial action coding system. Palo Alto, CA: Consulting Psychologists Press.

    Google Scholar 

  24. Ekman, P., & Friesen, W. V. (1982). Felt, false and miserable smiles. Journal of Nonverbal Behavior, 6, 238–252.

    Article  Google Scholar 

  25. Ekman, P., Friesen, W. V., & O’Sullivan, M. (1988). Smiles when lying. Journal of Personality and Social Psychology, 54, 414–420.

    PubMed  Article  Google Scholar 

  26. Forbes, R. J., & Jackson, P. R. (1980). Non-verbal behavior and the outcome of selection interviews. Journal of Occupational Psychology, 53, 65–72.

    Google Scholar 

  27. Forgas, J. P. (1985). Interpersonal behaviour: The psychology of social interaction. Sydney, Oxford, New York: Pergamon Press.

    Google Scholar 

  28. Hess, U. (2001). The communication of emotion. In A. Kaszniak (Ed.), Emotions, qualia and consciousness (pp. 397–409). Singapore: World Scientific Publishing.

    Google Scholar 

  29. Hess, U., Adams, R. B., & Kleck, R. E. (2005). Who may frown and who should smile? Dominance, affiliation, and the display of happiness and anger. Cognition and Emotion, 19, 515–536.

    Article  Google Scholar 

  30. Hess, U., Blairy, S., & Kleck, R. E. (1998). The relationship between the intensity of emotional facial expressions and observers’ decoding. Journal of Nonverbal Behavior, 21, 241–257.

    Article  Google Scholar 

  31. Hess, U., Blairy, S., & Kleck, R. E. (2000). The influence of facial emotion displays, gender, and ethnicity on judgments of dominance and affiliation. Journal of Nonverbal Behavior, 24, 265–283.

    Article  Google Scholar 

  32. Hess, U., & Kleck, R. E. (1990). Differentiating emotion elicited and deliberate emotional facial expressions. European Journal of Social Psychology, 20, 369–385.

    Article  Google Scholar 

  33. Imada, A. S., & Hakel, M. D. (1977). Influence of nonverbal communication and rater proximity on impressions and decisions in simulated employment interviews. Journal of Applied Psychology, 62, 295–300.

    Article  Google Scholar 

  34. Kamachi, M., Bruce, V., Mukaida, S., Gyoba, J., Yoshikawa, S., & Akamatsu, S. (2001). Dynamic properties influence the perception of facial expressions. Perception, 30, 875–887.

    PubMed  Article  Google Scholar 

  35. Kerlow, I. V. (2004). The art of 3D computer animation and effects (3rd ed.). New Jersey: John Wiley & Sons.

    Google Scholar 

  36. Koda, T., & Maes, P. (1996). Agents with faces: The effects of personification of agents. Proceedings of the 5th IEEE International Workshop on Robot and Human Communication (RO-MAN’96) (pp. 189–194) Japan.

  37. Krämer, N. C., Tietz, B., & Bente, G. (2003). Effects of embodied interface agents and their gestural activity. In R. Aylett, D. Ballin, T. Rist, & J. Rickel (Eds.), 4th International Working Conference on Intelligent Virtual Agents. Hamburg: Springer.

    Google Scholar 

  38. Krumhuber, E., & Kappas, A. (2005). Moving smiles: The role of dynamic components for the perception of the genuineness of smiles. Journal of Nonverbal Behavior, 29, 3–24.

    Article  Google Scholar 

  39. Krumhuber, E., Manstead, A. S. R., Cosker, D., Marshall, D., Rosin, P. L., & Kappas, A. (2007a). Facial dynamics as indicators of trustworthiness and cooperative behavior. Emotion, 7, 730–735.

    PubMed  Article  Google Scholar 

  40. Krumhuber, E., Manstead, A. S. R., & Kappas, A. (2007b). Temporal aspects of facial displays in person and expression perception: The effects of smile-dynamics, head-tilt and gender. Journal of Nonverbal Behavior, 31, 39–56.

    Article  Google Scholar 

  41. Lander, K., Christie, F., & Bruce, V. (1999). The role of movement in the recognition of famous faces. Memory and Cognition, 27, 974–985.

    Google Scholar 

  42. Rasmussen, K. G. (1984). Nonverbal behavior, verbal behavior, resumé credentials, and selection interview outcomes. Journal of Applied Psychology, 69, 551–556.

    Article  Google Scholar 

  43. Riggio, R. E., & Throckmorton, B. (1988). The relative effects of verbal and nonverbal behavior, appearance, and social skills on evaluations made in hiring interviews. Journal of Applied Social Psychology, 18, 331–348.

    Article  Google Scholar 

  44. Sato, W., & Yoshikawa, S. (2004). The dynamic aspects of emotional facial expressions. Cognition and Emotion, 18, 701–710.

    Article  Google Scholar 

  45. Schmidt, K. L., Ambadar, Z., Cohn, J. F., & Reed, L. I. (2006). Movement differences between deliberate and spontaneous facial expressions: Zygomaticus major action in smiling. Journal of Nonverbal Behavior, 30, 37–52.

    Article  PubMed  Google Scholar 

  46. Shrout, P. E., & Fiske, D. W. (1981). Nonverbal behaviors and social evaluation. Journal of Personality, 49, 115–128.

    Article  Google Scholar 

  47. Takács, B., & Kiss, B. (2003). The virtual human interface: A photorealistic digital human. IEEE Computer Graphics and Applications, 23, 38–45.

    Article  Google Scholar 

  48. Wallraven, C., Breidt, M., Cunningham, D. W., & Bülthoff, H. H. (2005). Psychophysical evaluation of animated facial expressions. Proceedings of the 2nd Symposium on Applied Perception in Graphics and Visualization. (pp 17–24). ACM Press: New York, NY, USA.

  49. Wehrle, T., Kaiser, S., Schmidt, S., & Scherer, K. R. (2000). Studying the dynamics of emotional expression using synthesized facial muscle movements. Journal of Personality and Social Psychology, 78, 105–119.

    PubMed  Article  Google Scholar 

  50. Weiss, F., Blum, G. S., & Gleberman, L. (1987). Anatomically based measurement of facial expression in simulated versus hypnotically induced affect. Motivation and Emotion, 11, 67–81.

    Article  Google Scholar 

  51. Wiberg, C., & Wiberg, M. (2001). Configuring social agents. In Proceedings of Conference of Universal Accessability in Human Computer Interaction, New Orleans.

  52. Young, D. M., & Beier, E. G. (1977). The role of applicant nonverbal communication in the employment interview. Journal of Employment Counseling, 14, 154–165.

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank the three human posers who participated in this research, Geoff Thomas and two anonymous reviewers for comments on previous versions of this paper.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Eva Krumhuber.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Krumhuber, E., Manstead, A.S.R., Cosker, D. et al. Effects of Dynamic Attributes of Smiles in Human and Synthetic Faces: A Simulated Job Interview Setting. J Nonverbal Behav 33, 1–15 (2009). https://doi.org/10.1007/s10919-008-0056-8

Download citation

Keywords

  • Facial expression
  • Smile
  • Dynamics
  • Job
  • Social perception
  • Decision