Skip to main content
SpringerLink
Log in

Applying Emotion Recognition to Graphic Design Research

  • Chapter
  • First Online:
Emotional Engineering, Vol.7

Abstract

We derive useful information about the feelings and inner states of other human beings by looking at the expressions on their faces. In this study, we conducted an experiment on the identification of emotions based on facial expression recognition to assess the participants’ emotional responses to different types of graphics of Chinese dragons. We used a paired sample t-test to analyze valid data collected from 112 participants. The results of our analysis indicate that there were differences between the participants’ expressions for a neutral response, happiness, sadness, and disgust. The effects of generalizing the participants’ facial expressions and associated verbalizations indicated that personalized designs aroused positive emotions, while the participants had more ideas associated with Chinese dragons when presented with realistic drawings. Furthermore, we found that the participants’ facial expressions were consistent with their verbal responses. This research demonstrates the importance and practicality of using facial expression recognition to assess peoples’ emotional responses to different graphic styles. Our results will help designers to build emotional communication with consumers and link their designs to consumers’ interests when promoting marketing activities.

The English in this document has been checked by at least two professional editors, both native speakers of English. For a certificate, see http://www.textcheck.com/certificate/f9ptvj.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aggarwal P, Mcgill AL (2007) Is that car smiling at me? Schema congruity as a basis for evaluating anthropomorphized products. J Consum Res 34:468–479

    Article  Google Scholar 

  2. Asthana HS (2012) Facial asymmetry in expression of emotion: a neurocultural perspective. Concept Publishing Company, New Delhi

    Google Scholar 

  3. Brunswik E (1955) Representative design and probabilistic theory in a functional psychology. Psychol Rev 62:193–217

    Article  Google Scholar 

  4. Carretié L, Hinojosa JA, Albert J, Mercado F (2006) Neural response to sustained affective visual stimulation using an indirect task. Exp Brain Res 174:630–637

    Article  Google Scholar 

  5. Carretié L, Martín-Loeches M, Hinojosa JA, Mercado F (2001) Emotion and attention interaction studied through event-related potentials. J Cogn Neurosci 13:1109–1128

    Article  Google Scholar 

  6. Cohen AS, Morrison SC, Callaway DA (2013) Computerized facial analysis for understanding constricted/blunted affect: Initial feasibility, reliability, and validity data. Schizophr Res 148:111–116

    Article  Google Scholar 

  7. Cuthbert BN, Schupp HT, Bradley MM, Birbaumer N, Lang PJ (2000) Brain potentials in affective picture processing: covariation with autonomic arousal and affective report. Biol Psychol 52:95–111

    Article  Google Scholar 

  8. Danner L, Haindl S, Joechl M, Duerrschmid K (2014) Facial expressions and autonomous nervous system responses elicited by tasting different juices. Food Res Int 64:81–90

    Article  Google Scholar 

  9. Delplanque S, Silvert L, Hot P, Rigoulot S, Sequeira H (2006) Arousal and valence effects on event-related P3a and P3b during emotional categorization. Int J Psychophysiol 60:315–322

    Article  Google Scholar 

  10. Dunn SW, Barban AM (1986) Advertising: its role in modern marketing. Dryden Press, Fort Worth

    Google Scholar 

  11. Ekman P (2003) Emotions revealed—recognizing faces and feelings to improve communication and emotional life. Times Books, New York

    Google Scholar 

  12. Ekman P, Friesen WV (1978) Facial action coding system: a technique for the measurement of facial movement. Consulting Psychologists Press, Palo Alto

    Google Scholar 

  13. Ekman P, Friesen WV, Hager JC (2002a) Facial action coding system—investigator’s guide. Research Nexus, Salt Lake City

    Google Scholar 

  14. Ekman P, Friesen WV, Hager JC (2002b) Facial action coding system—the manual. Research Nexus (a subsidiary of Network Information Research Corporation), Salt Lake City

    Google Scholar 

  15. Essa IA, Pentland AP (1997) Coding, analysis, interpretation, and recognition of facial expressions. IEEE Trans Pattern Anal Mach Intell 19:757–763

    Article  Google Scholar 

  16. Fridlund AJ (2014) Human facial expression: an evolutionary view. Academic Press, London

    Google Scholar 

  17. Hazlett R (2006) Measuring emotional valence during interactive experience: boys at video game play. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM Press, New York

    Google Scholar 

  18. Heath R, Brandt D, Nairn A (2006) Brand relationships: strengthened by emotion, weakened by attention. J Advert Res 46:410–419

    Article  Google Scholar 

  19. Heath R, Hyder P (2005) Measuring the hidden power of emotive advertising. J Advert Res 47:467–486

    Google Scholar 

  20. Hill D (2010) Emotionomics: leveraging emotions for business success. Kogan Page, London

    Google Scholar 

  21. Hirschman EC, Holbrook MB (1982) Hedonic consumption: emerging concepts, methods and propositions. J Market 46:92–101

    Article  Google Scholar 

  22. Hot P, Saito Y, Mandai O, Kobayashi T, Sequeira H (2006) An ERP investigation of emotional processing in European and Japanese individuals. Brain Res 1122:171–178

    Article  Google Scholar 

  23. I B, Lin YC (2006) Eye movements in image viewing processing. J Design 11:59–80

    Google Scholar 

  24. Juslin PN, Persson RS (2002) Emotional communication. In: Parncutt R, McPherson G (eds) The science and psychology of music performance: creative strategies for teaching and learning. Oxford University Press, Oxford

    Google Scholar 

  25. Keil A, Bradley MM, Hauk O, Rockstroh B, Elbert T, Lang PJ (2002) Large-scale neural correlates of affective picture processing. Psychophysiology 39:641–649

    Article  Google Scholar 

  26. Kemper TD, Ekman P (1981) The face of man: expressions of universal emotions in a New Guinea village. Contemp Sociol 10(4):521

    Article  Google Scholar 

  27. Kirk J, Jahoda A (2009) The role of emotional imagery and somatosensory amplification in atypical chest pain in patients with angina pectoris: a single-case experimental design. J Cardiopulmonary Rehabil Prevent 29:121–125

    Article  Google Scholar 

  28. Lajevardi SM, Wu HR (2012) Facial expression recognition in perceptual color space. IEEE Trans Image Process 21:3721–3733

    Article  MathSciNet  Google Scholar 

  29. Lau BT (2009) Portable real time needs expression for people with communication disabilities. In: Versick D (ed) Proceedings of communication in computer and information science: intelligent interactive assistance and mobile multimedia computing. Springer, Berlin, pp 85–95

    Google Scholar 

  30. Loijens L, Krips O (2013) FaceReader methodology. Noldus Information Technology, Wageningen

    Google Scholar 

  31. Mahike S, Minge M, Thüring M (2006) Measuring multiple components of emotions in interactive contexts. In: Work in progress CHI 2006. ACM Press, New York

    Google Scholar 

  32. Mahrabian A (1981) Silent messages. Wadsworth, Belmont

    Google Scholar 

  33. Matsunaga M, Isowa T, Kimura K, Miyakoshi M, Kanayama N, Murakami H, Fukuyama S, Shinoda J, Yamada J, Konagaya T, Kaneko H, Ohira H (2009) Associations among positive mood, brain, and cardiovascular activities in an affectively positive situation. Brain Res 1263:93–103

    Article  Google Scholar 

  34. Mitterschiffthaler MT, Fu CHY, Dalton JA, Andrew CM, Williams SCR (2007) A functional MRI study of happy and sad affective states induced by classical music. Human Brain Mapp 28:1150–1162

    Article  Google Scholar 

  35. Motterlini M (2008) Economia emotiva: Che cosa si nasconde dietro i nostri conti quotidiani. Milano, Rizzoli

    Google Scholar 

  36. Norman DA (2004) Emotional design: why we love (or hate) everyday things. Basic Books, New York

    Book  Google Scholar 

  37. Norris JS (1990) Advertising. Prentice Hall, New Jersey

    Google Scholar 

  38. Olofsson JK, Nordin S, Sequeira H, Polich J (2008) Affective picture processing: an integrative review of ERP findings. Biol Psychol 77:247–265

    Article  Google Scholar 

  39. Ozkaramanli D, Desmet PMA (2012) I knew i shouldn’t, yet i did it again! Emotion-driven Design as a means to motivate subjective well-being. Int J Design 6:27–39

    Google Scholar 

  40. Pantic M, Patras I (2006) Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences. IEEE Trans Syst Man Cybern Part B 36:433–449

    Article  Google Scholar 

  41. Planalp S (1999) Communicating emotion: social, moral, and cultural processes. Cambridge University Press, New York

    Book  Google Scholar 

  42. Pollak SD, Messner M, Kistler DJ, Cohn JF (2009) Development of perceptual expertise in emotion recognition. Cognition 110:242–247

    Article  Google Scholar 

  43. Pos OD, Green-Armytage P (2007) Facial expression, colours and basic emotions. Colour Design Creativity 1(2):1–20

    Google Scholar 

  44. Ray A, Chakrabarti A (2012) Design and implementation of affective e-learning strategy based on facial emotion recognition. In: Satapathy SC, Avadhani PS, Abraham A (eds) International conference on information systems design and intelligent applications, Visakhapatnam, India, pp 613–622

    Google Scholar 

  45. Rayner K (1998) Eye movements in reading and information processing: 20 years of research. Psychol Bull 124:372–422

    Article  Google Scholar 

  46. Reeve J, Nix G (1997) Expressing intrinsic motivation through acts of exploration and facial displays of interest. Motiv Emot 21:237–250

    Article  Google Scholar 

  47. Ricoeur P (1977) The rule of metaphor: multi-disciplinary studies of the creation of meaning in language. University of Toronto Press, Toronto

    Google Scholar 

  48. Rinn WE (1984) The neuropsychology of facial expression: a review of the neurological and psychological mechanisms for producing facial expressions. Psychol Bull 95:52–77

    Article  Google Scholar 

  49. Schachter S, Singer JE (1962) Cognitive, social, and physiological determinants of emotional state. Psychol Rev 69:379–399

    Article  Google Scholar 

  50. Scherer KR, Clark-Polner E, Mortillaro M (2011) In the eye of the beholder? Universality and cultural specificity in the expression and perception of emotion. Int J Psychol 46:401–435

    Article  Google Scholar 

  51. Song KT, Han MJ, Hong JW (2010) Online learning design of an image-based facial expression recognition system. Intel Serv Robot 3:151–162

    Article  Google Scholar 

  52. Wolf B (2005) Brunswik’s original lens model. Available: http://www.brunswik.org/notes/WolfOriginalLens2005.pdf. Accessed 20 May 2013

  53. Yu CY, Ko CH (2015) The application of facial expression recognition to the evaluation of stylized images. J Design Sci 18

    Google Scholar 

  54. Yu CY, Ko CH (2017) Applying FaceReader to recognize consumer emotions in graphic styles. Proc CIRP 60:104–109

    Article  Google Scholar 

  55. Zaman B, Shrimpton-Smith T (2006) The FaceReader: measuring instant fun of use. In: Nordic conference on human-computer interaction 2006, Olso, Norway

    Google Scholar 

  56. Zhan Y, Zhou G (2007) Facial expression recognition based on hybrid features and fusing discrete HMM. In: Shumaker R (ed) Virtual reality, HCII 2007, (LNCS 4563). Springer, Berlin

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Applied Arts, Fu Jen Catholic University, No. 510, Zhongzheng Rd., Xinzhuang District, New Taipei City, Taiwan

    Chia-yin Yu

  2. Department of Design, National Taiwan University of Science and Technology, Taipei City, Taiwan

    Chih-hsiang Ko

Authors
  1. Chia-yin Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chih-hsiang Ko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chia-yin Yu .

Editor information

Editors and Affiliations

  1. System Design and Management Research Institute, Keio University, Minato-ku, Tokyo, Japan

    Shuichi Fukuda

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Yu, Cy., Ko, Ch. (2019). Applying Emotion Recognition to Graphic Design Research. In: Fukuda, S. (eds) Emotional Engineering, Vol.7. Springer, Cham. https://doi.org/10.1007/978-3-030-02209-9_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-02209-9_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-02208-2

  • Online ISBN: 978-3-030-02209-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation