Skip to main content
SpringerLink
Log in

Emotional Debiasing Explanations for Decisions in HCI

  • Conference paper
  • First Online:
Artificial Intelligence in HCI (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14050))

Included in the following conference series:

Abstract

Emotions play an important role in human decision-making. However, first approaches to incorporating knowledge of this influence into AI-based decision support systems are only very recent. Accordingly, our target is to develop an interactive intelligent agent that is capable of explaining the recommendations of AI-systems while taking emotional constraints into account. This article addresses the following research questions based on the emotions of happiness and anxiety: (1) How do induced emotions influence risk propensity in HCI? (2) To what extent does the explanation strategy influence the human explanation recipient in a lottery choice? (3) How well can an HCI system estimate the emotional state of the human? Our results showed that (1) our emotion induction strategy was successful. However, the trend took the opposite direction of ATF predictions. (2) Our explanation strategy yielded a change in the risk decision in only 26% of the participants; in some cases, participants even changed their selection in the opposite direction. (3) Emotion recognition from facial expressions did not provide sufficient indications of the emotional state - because of head position and a lack of emotional display - but heart rate showed significant effects of emotion induction in the expected direction. Importantly, in individual cases, the dynamics of facial expressions followed the expected path. We concluded that (1) more differentiated explanation strategies are needed, and that temporal dynamics may play an important role in the explanation process, and (2) that a more interactive setting is required to elicit more emotional cues that can be used to adapt the explanation strategy accordingly.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Angie, A.D., Connelly, S., Waples, E.P., Kligyte, V.: The influence of discrete emotions on judgement and decision-making: a meta-analytic review. Cogn. Emotion 25(8), 1393–1422 (2011)

    Article  Google Scholar 

  2. Azarbarzin, A., Ostrowski, M., Hanly, P., Younes, M.: Relationship between arousal intensity and heart rate response to arousal. Sleep 37(4), 645–653 (2014)

    Article  Google Scholar 

  3. Bradley, M.M., Lang, P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49–59 (1994)

    Article  Google Scholar 

  4. Chaminade, T., Zecca, M., Blakemore, S.J., Takanishi, A., Frith, C.D., Micera, S., Dario, P., Rizzolatti, G., Gallese, V., Umiltà, M.A.: Brain response to a humanoid robot in areas implicated in the perception of human emotional gestures. PLoS ONE 5(7), e11577 (2010)

    Article  Google Scholar 

  5. Critchley, H.D., Rotshtein, P., Nagai, Y., O’Doherty, J., Mathias, C.J., Dolan, R.J.: Activity in the human brain predicting differential heart rate responses to emotional facial expressions. Neuroimage 24(3), 751–762 (2005)

    Article  Google Scholar 

  6. Crivelli, C., Fridlund, A.: Facial displays are tools for social influence. Trends Cognitive Sci. 22(5) (2018)

    Google Scholar 

  7. Ekman, P.: Universal and cultural differences in facial expressions of emotions. Nebraksa Symposium Motivation 19, 207–283 (1971)

    Google Scholar 

  8. Elliott, M.V., Johnson, S.L., Pearlstein, J.G., Lopez, D.E.M., Keren, H.: Emotion-related impulsivity and risky decision-making: a systematic review and meta-regression. Clinical Psychol. Rev., 102232 (2022)

    Google Scholar 

  9. Eyben, F., Wöllmer, M., Schuller, B.: Opensmile: the Munich versatile and fast open-source audio feature extractor. In: Proceedings of the 18th ACM International Conference on Multimedia, pp. 1459–1462 (2010)

    Google Scholar 

  10. Franke, T., Attig, C., Wessel, D.: A personal resource for technology interaction: development and validation of the affinity for technology interaction (ati) scale. Int. J. Hum.-Comput. Interact. 35(6), 456–467 (2019)

    Article  Google Scholar 

  11. Fredrickson, B.L.: Positive emotions broaden and build. In: Advances in Experimental Social Psychology, vol. 47, pp. 1–53. Elsevier (2013)

    Google Scholar 

  12. Hess, U., Banse, R., Kappas, A.: The intensity of facial expression is determined by underlying affective state and social situation. J. Personlaity Soc. Psychol. 69(2), 280–288 (1995)

    Article  Google Scholar 

  13. Holt, C.A., Laury, S.K.: Risk aversion and incentive effects. Am. Econ. Rev. 92(5), 1644–1655 (2002)

    Article  Google Scholar 

  14. Jaiswal, S., Virmani, S., Sethi, V., De, K., Roy, P.P.: An intelligent recommendation system using gaze and emotion detection. Multimed. Tools Appl. 78, 14231–14250 (2019)

    Article  Google Scholar 

  15. Kensinger, E.A.: Remembering the details: effects of emotion. Emot. Rev. 2(1), 99–113 (2009). https://doi.org/10.1177/1754073908100432

    Article  Google Scholar 

  16. Leiner, D. J.: Sosci survey. https://www.soscisurvey.de

  17. Lerner, J.S., Han, S., Keltner, D.: Feelings and consumer decision making: extending the appraisal-tendency framework. J. Consum. Psychol. 17(3), 181–187 (2007)

    Article  Google Scholar 

  18. Lerner, J.S., Keltner, D.: Beyond valence: toward a model of emotion-specific influences on judgement and choice. Cogn. Emotion 14(4), 473–493 (2000)

    Article  Google Scholar 

  19. Lerner, J.S., Keltner, D.: Fear, anger, and risk. J. Pers. Soc. Psychol. 81(1), 146 (2001)

    Article  Google Scholar 

  20. Lerner, J.S., Li, Y., Valdesolo, P., Kassam, K.S.: Emotion and decision making. Annu. Rev. Psychol. 66, 799–823 (2015)

    Article  Google Scholar 

  21. Lerner, J.S., Tiedens, L.Z.: Portrait of the angry decision maker: how appraisal tendencies shape anger’s influence on cognition. J. Behav. Decis. Mak. 19(2), 115–137 (2006)

    Article  Google Scholar 

  22. Lütkebohle, I., et al.: The bielefeld anthropomorphic robot head “flobi”. In: 2010 IEEE International Conference on Robotics and Automation, pp. 3384–3391 (2010). https://doi.org/10.1109/ROBOT.2010.5509173

  23. Mills, C., D’Mello, S.: On the validity of the autobiographical emotional memory task for emotion induction. PLoS ONE 9(4), e95837 (2014)

    Article  Google Scholar 

  24. Moscato, V., Picariello, A., Sperlí, G.: An emotional recommender system for music. IEEE Intell. Syst. 36(5), 57–68 (2021). https://doi.org/10.1109/MIS.2020.3026000

    Article  Google Scholar 

  25. Rosenthal-von der Pütten, A.M., Krämer, N.C., Hoffmann, L., Sobieraj, S., Eimler, S.C.: An experimental study on emotional reactions towards a robot. Int. J. Soc. Robot. 5(1), 17–34 (2013)

    Google Scholar 

  26. Rabin, M., Thaler, R.H.: Anomalies: risk aversion. J. Econ. Perspectives 15(1), 219–232 (2001)

    Article  Google Scholar 

  27. Rammstedt, B., Kemper, C.J., Klein, M.C., Beierlein, C., Kovaleva, A.: Big five inventory (bfi-10). Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS) (2014). https://doi.org/10.6102/zis76,https://zis.gesis.org/DoiId/zis76

  28. Ribeiro, F.S., Santos, F.H., Albuquerque, P.B., Oliveira-Silva, P.: Emotional induction through music: measuring cardiac and electrodermal responses of emotional states and their persistence. Front. Psychol. 10, 451 (2019)

    Article  Google Scholar 

  29. Riedl, M.O.: Human-centered artificial intelligence and machine learning. Hum. Behav. Emerg. Technol. 1(1), 33–36 (2019)

    Article  Google Scholar 

  30. Rohlfing, K.J., et al.: Explanation as a social practice: toward a conceptual framework for the social design of AI systems. IEEE Trans. Cogn. Dev. Syst. 13(3), 717–728 (2020)

    Article  Google Scholar 

  31. Ruiz-Belda, M.A., Fernández-Dols, J.M., Carrera, P., Barchard, K.: Spontaneous facial expressions of happy bowlers and soccer fans. Cogn. Emot. 17(2), 315–326 (2003)

    Article  Google Scholar 

  32. Schulreich, S., Gerhardt, H., Heekeren, H.R.: Incidental fear cues increase monetary loss aversion. Emotion 16(3), 402 (2016)

    Article  Google Scholar 

  33. Serengil, S.I., Ozpinar, A.: Hyperextended lightface: a facial attribute analysis framework. In: 2021 International Conference on Engineering and Emerging Technologies (ICEET), pp. 1–4. IEEE (2021). https://doi.org/10.1109/ICEET53442.2021.9659697

  34. Siedlecka, E., Denson, T.F.: Experimental methods for inducing basic emotions: a qualitative review. Emot. Rev. 11(1), 87–97 (2019)

    Article  Google Scholar 

  35. Smith, C.A., Ellsworth, P.C.: Patterns of cognitive appraisal in emotion. J. Pers. Soc. Psychol. 48(4), 813 (1985)

    Article  Google Scholar 

  36. Spielberger, C.D.: Manual for the state-trait anxietry, inventory. Consulting Psychologist (1970)

    Google Scholar 

  37. Thiruchselvam, R., Blechert, J., Sheppes, G., Rydstrom, A., Gross, J.J.: The temporal dynamics of emotion regulation: an eeg study of distraction and reappraisal. Biol. Psychol. 87(1), 84–92 (2011). https://doi.org/10.1016/j.biopsycho.2011.02.009. https://www.sciencedirect.com/science/article/pii/S0301051111000391

  38. Toisoul, A., Kossaifi, J., Bulat, A., Tzimiropoulos, G., Pantic, M.: Estimation of continuous valence and arousal levels from faces in naturalistic conditions. Nature Mach. Intell. (2021). https://www.nature.com/articles/s42256-020-00280-0

  39. Wang, D., Yang, Q., Abdul, A., Lim, B.Y.: Designing theory-driven user-centric explainable AI. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–15 (2019)

    Google Scholar 

  40. Yang, Q., Zhou, S., Gu, R., Wu, Y.: How do different kinds of incidental emotions influence risk decision making? Biol. Psychol. 154, 107920 (2020)

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank Maryam Alizadeh (Bielefeld University) and Anamaria Cubelic (Paderborn University) for their assistance with data collection in their master theses. The work described in this paper was supported by a grant from the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation): TRR 318/1 2021 - 438445824.

Author information

Authors and Affiliations

  1. Medical Assistance Systems, Medical School OWL, Bielefeld University, Universitätsstraße 25, 33615, Bielefeld, Germany

    Christian Schütze & Birte Richter

  2. Center for Cognitive Interaction Technology, CITEC, Bielefeld University, Universitätsstraße 25, 33615, Bielefeld, Germany

    Christian Schütze & Birte Richter

  3. Faculty of Business Administration and Economics, Department of Management, Paderborn University, Warburger Str. 100, 33098, Paderborn, Germany

    Olesja Lammert & Kirsten Thommes

  4. Software Engineering for Cognitive Robots and Systems, University of Bremen, Am Fallturm 1, 128359, Bremen, Germany

    Britta Wrede

Authors
  1. Christian Schütze
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Olesja Lammert
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Birte Richter
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kirsten Thommes
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Britta Wrede
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christian Schütze .

Editor information

Editors and Affiliations

  1. Siemens Corporation, Princeton, NJ, USA

    Helmut Degen

  2. Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Stavroula Ntoa

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Schütze, C., Lammert, O., Richter, B., Thommes, K., Wrede, B. (2023). Emotional Debiasing Explanations for Decisions in HCI. In: Degen, H., Ntoa, S. (eds) Artificial Intelligence in HCI. HCII 2023. Lecture Notes in Computer Science(), vol 14050. Springer, Cham. https://doi.org/10.1007/978-3-031-35891-3_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35891-3_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35890-6

  • Online ISBN: 978-3-031-35891-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation