Skip to main content

Emoji Face Renderings: Exploring the Role Emoji Platform Differences have on Emotional Interpretation

Abstract

Emoji faces are ubiquitous and integrated into most people’s everyday (nonverbal) vernacular. Yet, we know little about how people interpret these characters in terms of their emotional content. Do people agree that an emoji face represents an individual emotion and that it is unique to a specific emotion? Are such representations similar across electronic platforms? The present study took a theoretical approach to address these questions by investigating shared agreement between emoji–emotion pairings across three electronic platforms (Apple, Android, and Samsung). Two hundred twenty-eight English-speaking adults completed an online survey that involved picking up to three emoji faces (presented from a common set from one of three platforms) for each of 10 emotions. They then indicated the strength of that relationship. We examined whether the intensities that participants gave to emoji–emotion pairings were specific and consistent across platforms. Our results showed limited shared agreement for the majority of emoji–emotion pairings, and significant variation as to which emotion category a “comparable” emoji belonged depending upon the viewed platform. We highlight the need for future emoji perception research to examine how different platform renderings of the same emoji might lead to miscommunication and interpretation discrepancies.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Notes

  1. 1.

    There were two exceptions to this rule: We decided to keep Samsung’s confounded-face emoji and the worried-face emoji even though these particular emojis had an embellishment. In these cases, we removed the embellishments with Photoshop.

  2. 2.

    A forced-choice task is less reflective of a natural choice task (e.g. participants are forced to assign one emoji face to an emotion category). Therefore, any “universal” findings might result from the restrictive design.

References

  1. Aviezer, H., Bentin, S., Dudarev, V., & Hassin, R. R. (2011). The automaticity of emotional face-context integration. Emotion,11(6), 1406.

    Article  Google Scholar 

  2. Aviezer, H., Hassin, R. R., Ryan, J., Grady, C., Susskind, J., Anderson, A., et al. (2008). Angry, disgusted, or afraid? Studies on the malleability of emotion perception. Psychological Science,19(7), 724–732.

    Article  Google Scholar 

  3. Aviezer, H., Trope, Y., & Todorov, A. T. (2012). Body cues, not facial expressions, discriminate between intense positive and negative emotions. Science,338(6111), 1225–1229. https://doi.org/10.1126/science.1224313.

    Article  PubMed  Google Scholar 

  4. Barrett, L. F. (2006a). Are emotions natural kinds? Perspectives on Psychological Science,1(1), 28–58. https://doi.org/10.1111/j.1745-6916.2006.00003.x.

    Article  PubMed  Google Scholar 

  5. Barrett, L. F. (2006b). Solving the emotion paradox: Categorization and the experience of emotion. Personality and Social Psychology Review,10(1), 20–46. https://doi.org/10.1207/s15327957pspr1001_2.

    Article  PubMed  Google Scholar 

  6. Barrett, L. F. (2011). Was Darwin wrong about emotional expressions? Current Directions in Psychological Science,20(6), 400–406. https://doi.org/10.1177/0963721411429125.

    Article  Google Scholar 

  7. Barrett, L. F. (2012). Emotions are real. Emotion,12(3), 413–429. https://doi.org/10.1037/a002755.

    Article  PubMed  Google Scholar 

  8. Barrett, L. F. (2014). The conceptual act theory: A précis. Emotion Review,6(4), 292–297.

    Article  Google Scholar 

  9. Barrett, L. F. (2017). How emotions are made: The secret life of the brain. Boston, MA: Houghton Mifflin Harcourt.

    Google Scholar 

  10. Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019). Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements. Psychological Science in the Public Interest,20(1), 1–68. https://doi.org/10.1177/1529100619832930.

    Article  PubMed  Google Scholar 

  11. Barrett, L. F., Lindquist, K. A., & Gendron, M. (2007). Language as context for the perception of emotion. Trends in Cognitive Sciences,11(8), 327–332.

    Article  Google Scholar 

  12. Brosch, T., Pourtois, G., & Sander, D. (2010). The perception and categorisation of emotional stimuli: A review. Cognition and Emotion,24(3), 377–400. https://doi.org/10.1080/02699930902975754.

    Article  Google Scholar 

  13. Calvo, M. G., & Nummenmaa, L. (2008). Detection of emotional faces: Salient physical features guide effective visual search. Journal of Experimental Psychology: General,137(3), 471–494.

    Article  Google Scholar 

  14. Carlson, J. R., & Zmud, R. W. (1999). Channel expansion theory and the experiential nature of media richness perceptions. Academy of Management Journal,42(2), 153–170.

    Google Scholar 

  15. Cha, Y. J. (2007). Identifying cross-cultural differences of emoticons in computer-mediated communication: A comparison of North American and South Korean emotional responses to emoticons. Thesis for the master of science in human-computer interaction, Indiana University. http://hdl.handle.net/1805/2351.

  16. Coelho, C. M., Cloete, S., & Wallis, G. (2009). The face-in-the-crowd effect: When angry faces are just cross. Journal of Vision,10(1), 7.

    Article  Google Scholar 

  17. Cordaro, D. T., Sun, R., Keltner, D., Kamble, S., Huddar, N., & McNeil, G. (2018). Universals and cultural variations in 22 emotional expressions across five cultures. Emotion,18(1), 75. https://doi.org/10.1037/emo0000302.

    Article  PubMed  Google Scholar 

  18. Cowen, A. S., & Keltner, D. (2017). Self-report captures 27 distinct categories of emotion bridged by continuous gradients. Proceedings of the National Academy of Sciences,114(38), E7900–E7909.

    Article  Google Scholar 

  19. de Waal, F. B. M. (2003). Darwin’s legacy and the study of primate visual communication. In P. Ekman, J. J. Campos, R. J. Davidson, & F. B. M. De Waal (Eds.), Emotions inside out: 130 years after Darwin’s the expression of the emotions in man and animals (pp. 7–31). New York, NY: The New York Academy of Sciences.

    Google Scholar 

  20. Derks, D., Bos, A. E., & Von Grumbkow, J. (2008). Emoticons in computer-mediated communication: Social motives and social context. Cyber Psychology and Behavior,11(1), 99–101.

    Article  Google Scholar 

  21. Ekman, P. (1992). Are there basic emotions? Psychological Review,99, 550–553.

    Article  Google Scholar 

  22. Ekman, P. (1993). Facial expression and emotion. American Psychologist,48, 384–392.

    Article  Google Scholar 

  23. Ekman, P. (2016). What scientists who study emotion agree about. Perspectives on Psychological Science,11(1), 31–34. https://doi.org/10.1177/1745691615596992.

    Article  PubMed  Google Scholar 

  24. Ekman, P., & Rosenberg, E. L. (Eds.). (1997). What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS). Oxford: Oxford University Press.

    Google Scholar 

  25. Emojipedia. (2018). Emojipediahome of Emoji meanings . Retrieved from http://emojipedia.org/.

  26. Esteves, F., Dimberg, U., & Öhman, A. (1994). Automatically elicited fear: Conditioned skin conductance responses to masked facial expressions. Cognition and Emotion,8(5), 393–413.

    Article  Google Scholar 

  27. Fox, E., Lester, V., Russo, R., Bowles, R. J., Pichler, A., & Dutton, K. (2000). Facial expressions of emotion: Are angry faces detected more efficiently? Cognition and Emotion,14(1), 61–92.

    Article  Google Scholar 

  28. Fugate, J. M., & Barrett, L. F. (2014). The role of language on the perception and experience. In T. Holtgraves (Ed.), The Oxford handbook of language and social psychology (pp. 28–294). New York, NY: Oxford University Press.

    Google Scholar 

  29. Fugate, J. M. B., & Franco, C. L. (solicited). A review of electronic facial expression research and its relationship to “basic” emotions. In Frontiers in psychology: Advances and obstacles in contemporary nonverbal communication research.

  30. Gudykunst, W. B. (1997). Cultural variability in communication: An introduction. Human Communication Research,24(4), 327–348. https://doi.org/10.1177/009365097024004001.

    Article  Google Scholar 

  31. Hansen, C. H., & Hansen, R. D. (1988). Finding the face in the crowd: An anger superiority effect. Journal of Personality and Social Psychology,54(6), 917.

    Article  Google Scholar 

  32. Izard, C. E. (1991). The psychology of emotions. New York, NY: Plenum Publishing.

    Book  Google Scholar 

  33. Izard, C. E. (1994). Innate and universal facial expressions: Evidence from developmental and cross-cultural research. Psychological Bulletin,115(2), 288–299.

    Article  Google Scholar 

  34. Izard, C. E. (2013). Human emotions. New York, NY: Springer.

    Google Scholar 

  35. Kelly, C. (2015). Do you know what I mean >:(: A linguistic study of the understanding of emoticons and emoji in text messages. Retrieved November 12, 2015 from http://www.diva-portal.org/.

  36. Keltner, D., Sauter, D., Tracy, J., & Cowen, A. (2019). Emotional expression: Advances in basic emotion theory. Journal of Nonverbal Behavior,43(2), 133–160.

    Article  Google Scholar 

  37. Kendall, M. G., & Smith, B. B. (1939). The problem of m rankings. The Annals of Mathematical Statistics,10(3), 275–287.

    Article  Google Scholar 

  38. Kennedy, P. (2014). Who made that emoticon? Retrieved August 26, 2014, from http://www.nytimes.com/2012/11/25/magazine/who-made-that-emoticon.html.

  39. Kupperbusch, C., Matsumoto, D., Kooken, K., Loewinger, S., Uchida, H., Wilson-Cohn, C., et al. (1999). Cultural influences on nonverbal expressions of emotion. In P. Philippot, R. S. Feldman, & E. J. Coats (Eds.), The social context of nonverbal behavior (pp. 17–44). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  40. Lebduska, L. (2014). Emoji, emoji, what for art thou? Retrieved November 12, 2014, from http://harlotofthearts.org/index.php/harlot/article.

  41. LeDoux, J. (1996). Emotional networks and motor control: A fearful view. Progress in Brain Research,107, 437–446.

    Article  Google Scholar 

  42. LeDoux, J. (2003). The emotional brain, fear, and the amygdala. Cellular and Molecular Neurobiology,23(4–5), 727–738.

    Article  Google Scholar 

  43. Lindquist, K. A., & Gendron, M. (2013). What’s in a word? Language constructs emotion perception. Emotion Review,5(1), 66–71. https://doi.org/10.1177/1754073912451351.

    Article  Google Scholar 

  44. Ling, R. (2010). Texting as a life phase medium. Journal of Computer-Mediated Communication,15(2), 277–292. https://doi.org/10.1111/j.1083-6101.2010.01520.

    Article  Google Scholar 

  45. Liu, D., & Yang, C. C. (2016). Media niche of electronic communication channels in friendship: A meta-analysis. Journal of Computer-Mediated Communication,21(6), 451–466. https://doi.org/10.1111/jcc4.12175.

    Article  Google Scholar 

  46. Lo, S. K. (2008). The nonverbal communication functions of emoticons in computer-mediated communication. Cyber Psychology and Behavior,11(5), 595–597. https://doi.org/10.1089/cpb.2007.0132.

    Article  Google Scholar 

  47. Lu, X., Ai, W., Liu, X., Li, Q., Wang, N., Huang, G., et al. (2016). Learning from the ubiquitous language: An empirical analysis of emoji usage of smartphone users. Journal of Cross-Cultural Psychology,20(1), 92–105. https://doi.org/10.1145/2971648.2971724.

    Article  Google Scholar 

  48. Matsumoto, D. (1989a). Cultural influences on the perception of emotion. Journal of Cross-Cultural Psychology,20(1), 92–105. https://doi.org/10.1177/0022022189201006.

    Article  Google Scholar 

  49. Matsumoto, D. (1989b). Face, culture, and judgments of anger and fear: Do the eyes have it? Journal of Nonverbal Behavior,13(3), 171–188. https://doi.org/10.1007/BF00987048.

    Article  Google Scholar 

  50. Matsumoto, D., Keltner, D., Shiota, M. N., Sullivan, M., & Frank, M. (2008). Facial expressions of emotion. In M. Lewis, J. M. Haviland-Jones, & L. F. Barrett (Eds.), Handbook of emotions (3rd ed., pp. 211–234). New York, NY: The Guildford Press.

    Google Scholar 

  51. Miller, H., Thebault-Spieker, J., Chang, S., Johnson, I., Terveen, L., & Hecht, B. (2016). Blissfully happy or ready to fight: Varying interpretations of emoji. In Tenth international AAAI conference on web and social media. International conference on weblog and social media (pp. 259–268).

  52. Nakano, M. (2015). Why and how I created emoji—IGNITION. Retrieved November 11, 2015, from http://ignition.co/105.

  53. Novak, P., Smailović, J., Sluban, B., & Mozetič, I. (2015). Sentiment of emoji. PLOS ONE,10(12), e0144296. https://doi.org/10.1371/journal.pone.0144296.

    Article  Google Scholar 

  54. Öhman, A. (2005). The role of the amygdala in human fear: Automatic detection of threat. Psychoneuroendocrinology,30(10), 953–958. https://doi.org/10.1016/j.psyneuen.2005.03.019.

    Article  PubMed  Google Scholar 

  55. Oleszkiewicz, A., Frackowiak, T., Sorokowska, A., & Sorokowski, P. (2017). Children can accurately recognize facial emotions from emoticons. Computers in Human Behavior,76, 372–377. https://doi.org/10.1016/j.chb.2017.07.040.

    Article  Google Scholar 

  56. Parr, L. A., Waller, B. M., Vick, S. J., & Bard, K. A. (2007). Classifying chimpanzee facial expressions using muscle action. Emotion,7(1), 172. https://doi.org/10.1037/1528-3542.7.1.172.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Plutchik, R. (1980). A general psychoevolutionary theory of emotion. In R. Plutchik & H. Kellerman (Eds.), Theories of emotion (pp. 3–33). New York: NY, Academic Press.

    Chapter  Google Scholar 

  58. Quintana, L. (2014). Cultural differences in style and usage of emoticons in computer-mediated communication. International Conference on Weblog and Social Media,16, 1–35.

    Google Scholar 

  59. Reynolds, R. M., Novotny, E., Lee, J., Roth, D., & Bente, G. (2019). Ambiguous bodies: The role of displayed arousal in emotion [mis] perception. Journal of Nonverbal Behavior,31, 1–20.

    Google Scholar 

  60. Riordan, M. A. (2017). Emojis as tools for emotion work: Communicating affect in text messages. Journal of Language and Social Psychology,36(5), 549–567.

    Article  Google Scholar 

  61. Sauter, D. A., LeGuen, O., & Haun, D. (2011). Categorical perception of emotional facial expressions does not require lexical categories. Emotion,11(6), 1479–1483.

    Article  Google Scholar 

  62. SwiftKey. (2015). Most-used emoji revealed: Americans love skulls, Brazilians love cats, the French love hearts [blog]. http://swiftkey.com/en/blog/americans-love-skulls-brazilians-love-cats-swiftkey-emoji-meanings-report/.

  63. Thompson, D., & Filik, R. (2016). Sarcasm in written communication: Emoticons are efficient markers of intention. Journal of Computer-Mediated Communication,21(2), 105–120. https://doi.org/10.1111/jcc4.12156.

    Article  Google Scholar 

  64. Tigwell, G. W., & Flatla, D. R. (2016). Oh that’s what you meant!: Reducing emoji misunderstanding. In Proceedings of the 18th international conference on human-computer interaction with mobile devices and services adjunct (pp. 859–866).

  65. Tipples, J., Atkinson, A. P., & Young, A. W. (2002). The eyebrow frown: A salient social signal. Emotion,2(3), 288–296.

    Article  Google Scholar 

  66. Tomkins, S. S. (1962). Affect, imagery, consciousness. The positive affects (Vol. 1, pp. 120–131). New York, NY: Springer.

    Google Scholar 

  67. Toratani, Y., & Hirayama, M. J. (2011). Psychological analysis of emoticons used for e-mails on cellular phones. International Conference on Mobile IT Convergence,2(8), 49–53.

    Google Scholar 

  68. Troiano, G., & Nante, N. (2018). Emoji: What does the scientific literature say about them? A new way to communicate in the 21th century. Journal of Human Behavior in the Social Environment,28(4), 528–533.

    Article  Google Scholar 

  69. Trope, Y. (1986). Identification and inferential processes in dispositional attribution. Psychological Review,93(3), 239.

    Article  Google Scholar 

  70. Unicode: Full Emoji List. (2018). Retrieved March 3, 2018, from https://unicode.org/emoji/charts/full-emoji-list.

  71. Van Hooff, J. A. R. M. (1962). Facial expressions in higher primates. Symposia of the Zoological Society of London,8, 97–125.

    Google Scholar 

  72. Williams, M. A., & Mattingley, J. B. (2006). Do angry men get noticed? Current Biology,16(11), 402–404.

    Article  Google Scholar 

  73. Wolny, W. (2016). Sentiment analysis of Twitter data using emoticons and emoji ideograms. Studia Ekonomiczne,296, 163–171.

    Google Scholar 

Download references

Acknowledgements

We would like to thank A. Scott McCauley for help with programming the survey. We would also like to thank Tatum Leclair and Maatangi Krishna who helped code the data. Finally, we would like to thank Aminda O’Hare and Mahzad Hojjat for assistance with the emoji selection.

Funding

This work was supported by the Office of Undergraduate Research at the University of Massachusetts-Dartmouth to Courtny Franco.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jennifer M. B. Fugate.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 1492 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Franco, C.L., Fugate, J.M.B. Emoji Face Renderings: Exploring the Role Emoji Platform Differences have on Emotional Interpretation. J Nonverbal Behav 44, 301–328 (2020). https://doi.org/10.1007/s10919-019-00330-1

Download citation

Keywords

  • Emoji faces
  • Electronic platforms
  • Electronic communication
  • Emotion
  • Emotion theory