Skip to main content
SpringerLink
Log in
SpringerLink
Log in

Measuring Physiological Markers of Stress During Conversational Agent Interactions

  • Chapter
  • First Online:
AI for Disease Surveillance and Pandemic Intelligence (W3PHAI 2021)

Part of the book series: Studies in Computational Intelligence ((SCI,volume 1013))

Included in the following conference series:

Abstract

Conversational agent (CA) technology is rapidly becoming ubiquitous. Understanding how CAs impact users on multiple levels, including physiology, thus becomes increasingly important. In this study, we examined the effects of a CA interaction on naive users’ physiological markers of stress i.e. heart rate (HR) and electrodermal activity (EDA). Participants (n = 21) prepared and executed a speech as part of a stressful interview, followed by a “Wizard-of-Oz” CA interaction. We expected the CA interactions to be mildly stressful. For a subset of participants with an initial resting period (n = 10), HR was elevated by 4.06 beats per minute (bpm) on average during the speech task, relative to the resting baseline. During the CA interaction however, HR was found to be 1.16 bpm lower on average. Moreover, HR and EDA values during the CA interaction were highly correlated with those during the resting period (Spearman’s rho: HR = 0.97, EDA = 0.96) with small differences (mean diff: HR = 1.16, EDA = 1.82). Contrary to initial expectations, users do not seem to exhibit a stress response during the CA interaction. We additionally performed similar analyses and compared our results with the Wearable Stress and Affect Detection (WESAD) dataset [1].

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Schmidt, P., Reiss, A., Duerichen, R., Marberger, C., Van Laerhoven, K.: Introducing WESAD, a multimodal dataset for wearable stress and affect detection. In: Proceedings of the 20th ACM International Conference on Multimodal Interaction, pp. 400–408 (2018)

    Google Scholar 

  2. Porcheron, M., Fischer, J. E., Reeves, S., Sharples, S.: Voice interfaces in everyday life. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–12 (2018)

    Google Scholar 

  3. Car, L.T., Dhinagaran, D.A., Kyaw, B.M., Kowatsch, T., Joty, S., Theng, Y.L., Atun, R.: Conversational agents in health care: Scoping review and conceptual analysis. J. Med. Internet Res. 22, e17158 (2020)

    Article  Google Scholar 

  4. Hobert, S., Meyer von Wolff, R.: Say hello to your new automated tutor—a structured literature review on pedagogical conversational agents. In: 14th International Conference on Wirtschaftsinformatik. Siegen (2019)

    Google Scholar 

  5. Mozer, T: Speech’s Evolving Role in Consumer Electronics... From Toys to Mobile. In: Mobile Speech and Advanced Natural Language Solutions, pp. 23–34. Springer (2013)

    Google Scholar 

  6. Cowan, B.R., Pantidi, N., Coyle, D., Morrissey, K., Clarke, P., Al-Shehri, S., Earley, D., Bandeira, N.: “What can I help you with?” Infrequent users’ experiences of intelligent personal assistants. In: Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services, pp. 1–12 (2017)

    Google Scholar 

  7. McTear, M.F.: Spoken dialogue technology: enabling the conversational user interface. Assoc. Comput. Mach. Comput. Surv. (ACM-CSUR). 34, 90–169 (2002)

    Google Scholar 

  8. Schroeder, J., Schroeder, M.: Trusting in machines: how mode of interaction affects willingness to share personal information with machines. In: Proceedings of the 51st Hawaii International Conference on System Sciences (2018)

    Google Scholar 

  9. Ciechanowski, L., Przegalinska, A., Magnuski, M., Gloor, P.: In the shades of the uncanny valley: an experimental study of human-chatbot interaction. Future Gener. Comput. Syst. 92, 539–548 (2019)

    Article  Google Scholar 

  10. Barreto, A., Zhai, J., Adjouadi, M.: Non-intrusive physiological monitoring for automated stress detection in human-computer interaction. In: International Workshop on Human-Computer Interaction, pp. 29–38. Springer (2007)

    Google Scholar 

  11. Lee, S., Ryu, H., Park, B., Yun, M.H.: Using physiological recordings for studying user experience: case of conversational agent-equipped TV. Int. J. Hum. Comput. Interact. 36, 815–827 (2020)

    Article  Google Scholar 

  12. Prendinger, H., Becker, C., Ishizuka, M.: A study in users’physiological response to an empathic interface agent. Int. J. Hum. Robot. 3, 371–391 (2006)

    Article  Google Scholar 

  13. Mori, J., Prendinger, H., Ishizuka, M.: Evaluation of an embodied conversational agent with affective behavior. In: Proceedings of the AAMAS03 Workshop on Embodied Conversational Characters as Individuals (2003)

    Google Scholar 

  14. Kirschbaum, C., Pirke, K.M., Hellhammer, D.H.: The ‘Trier Social Stress Test’-a tool for investigating psychobiological stress responses in a laboratory setting. Neuropsychobiol. 28, 76–81 (1993)

    Article  Google Scholar 

  15. Han, K.S., Kim, L., Shim, I.: Stress and sleep disorder. Exp. Neurobiol. 21, 141–150 (2012)

    Google Scholar 

  16. Kivimäki, M., Steptoe, A.: Effects of stress on the development and progression of cardiovascular disease. Natl. Rev. Cardiol. 15, 215–229 (2018)

    Article  Google Scholar 

  17. Weizenbaum, J.: ELIZA-a computer program for the study of natural language communication between man and machine. Commun. Assoc. Comput. Mach. (ACM) 9, 36–45 (1966)

    Google Scholar 

  18. Spänig, S., Emberger-Klein, A., Sowa, J.P., Canbay, A., Menrad, K., Heider, D.: The virtual doctor: an interactive clinical-decision-support system based on deep learning for non-invasive prediction of diabetes. Artif. Intell. Med. 100, 101706 (2019)

    Article  Google Scholar 

  19. Ghosh, S., Bhatia, S., Bhatia, A.: Quro: facilitating user symptom check using a personalised chatbot-oriented dialogue system. Stud. Health Technol. Inform. 252, 51–56 (2018)

    Google Scholar 

  20. Fitzpatrick, K.K., Darcy, A., Vierhile, M.: Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial. J. Med. Internet Res. Ment. Health 4, e29 (2017)

    Google Scholar 

  21. Galescu, L., Allen, J., Ferguson, G., Quinn, J., Swift, M.: Speech recognition in a dialog system for patient health monitoring. In: 2009 IEEE International Conference on Bioinformatics and Biomedicine Workshop, pp. 302–307. IEEE (2009)

    Google Scholar 

  22. Fadhil, A.: Beyond patient monitoring: conversational agents role in telemedicine and healthcare support for home-living elderly individuals. arXiv:1803.06000 (2018)

  23. Ferland, L., Li, Z., Sukhani, S., Zheng, J., Zhao, L., Gini, M. L.: Assistive AI for coping with memory loss. In: AAAI Workshops, pp. 431–434 (2018)

    Google Scholar 

  24. Foo, E.W., Lee, J.W., Compton, C., Ozbek, S., Holschuh, B.: User experiences of garment-based dynamic compression for novel haptic applications. In: Proceedings of the 23rd International Symposium on Wearable Computers, pp. 54–59 (2019)

    Google Scholar 

  25. Foo, E., Baker, J., Compton, C., Holschuh, B.: Soft robotic compression garment to assist novice meditators. In: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–8 (2020)

    Google Scholar 

  26. Bent, B., Goldstein, B.A., Kibbe, W.A., Dunn, J.P.: Investigating sources of inaccuracy in wearable optical heart rate sensors. Nat. Partn. J. Digit. Med. 3, 1–9 (2020)

    Google Scholar 

  27. van Lier, H.G., Pieterse, M.E., Garde, A., Postel, M.G., de Haan, H.A., Vollenbroek-Hutten, M.M.R., Schraagen, J.M., Noordzij, M.L.: A standardized validity assessment protocol for physiological signals from wearable technology: methodological underpinnings and an application to the E4 biosensor. Behav. Res. Methods, 1–23 (2019)

    Google Scholar 

  28. Mitchell, W.J., Ho, C.C., Patel, H., MacDorman, K.F.: Does social desirability bias favor humans? Explicit-implicit evaluations of synthesized speech support a new HCI model of impression management. Comput. Hum. Behav. 27, 402–412 (2011)

    Article  Google Scholar 

  29. Habler, F., Schwind, V., Henze, N.: Effects of Smart Virtual Assistants’ Gender and Language. In: Proceedings of Mensch and Computer 2019, pp. 469–473 (2019)

    Google Scholar 

  30. Thayer, J.F., Åhs, F., Fredrikson, M., Sollers, J.J., III., Wager, T.D.: A meta-analysis of heart rate variability and neuroimaging studies: implications for heart rate variability as a marker of stress and health. Neurosci. Biobehav. Rev. 36, 747–756 (2012)

    Google Scholar 

  31. R Core Team.: R: A language and environment for statistical computing. R Foundation for Statistical Computing (2020)

    Google Scholar 

  32. Richardson, P., McKenna, W., Bristow, M., Maisch, B., Mautner, B., O’Connell, J., Olsen, E., Thiene, G., Goodwin, J., Gyarfas, I., Martin, I., Nordet, P.: Report of the 1995 World Health Organization/International Society and Federation of Cardiology Task Force on the Definition and Classification of cardiomyopathies (1996). https://doi.org/10.1161/01.cir.93.5.841

  33. Barrios, L., Oldrati, P., Santini, S., Lutterotti, A.: Evaluating the accuracy of heart rate sensors based on photoplethysmography for in-the-wild analysis. In: Proceedings of the 13th EAI International Conference on Pervasive Computing Technologies for Healthcare, pp. 251–261 (2019)

    Google Scholar 

  34. Kotlyar, M., Brauer, L.H., al’Absi, M., Adson, D.E., Robiner, W., Thuras, P., Harris, J., Finocchi, M. E., Bronars, C.A., Candell, S., Hatsukami, D.K.: Effect of bupropion on physiological measures of stress in smokers during nicotine withdrawal. Pharmacol. Biochem. Behav. 83, 370–379 (2006)

    Google Scholar 

  35. Kelly, M.M., Tyrka, A.R., Anderson, G.M., Price, L.H., Carpenter, L.L.: Sex differences in emotional and physiological responses to the Trier Social Stress Test. J. Behav. Ther. Exp. Psychiatr. 39, 87–98 (2008)

    Article  Google Scholar 

  36. Sgoifo, A., Braglia, F., Costoli, T., Musso, E., Meerlo, P., Ceresini, G., Troisi, A.: Cardiac autonomic reactivity and salivary cortisol in men and women exposed to social stressors: relationship with individual ethological profile. Neurosci. Biobehav. Rev. 27, 179–188 (2003)

    Article  Google Scholar 

  37. Kudielka, B.M., Buske-Kirschbaum, A., Hellhammer, D.H., Kirschbaum, C.: Differential heart rate reactivity and recovery after psychosocial stress (TSST) in healthy children, younger adults, and elderly adults: the impact of age and gender. Int. J. Behav. Med. 11, 116–121 (2004)

    Article  Google Scholar 

  38. Pakhomov, S.V.S., Thuras, P.D., Finzel, R., Eppel, J., Kotlyar, M.: Using consumer-wearable technology for remote assessment of physiological response to stress in the naturalistic environment. Public Libr. Sci. One 15, e0229942 (2020)

    Google Scholar 

  39. Ollander, S., Godin, C., Campagne, A., Charbonnier, S.: A comparison of wearable and stationary sensors for stress detection. In: 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 004362–004366 (2016)

    Google Scholar 

  40. Yaghoubzadeh, R., Kramer, M., Pitsch, K., Kopp, S.: Virtual agents as daily assistants for elderly or cognitively impaired people. In: International Workshop on Intelligent Virtual Agents, pp. 79–91 (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Minnesota, Minneapolis, MN, USA

    Shreya Datar, Libby Ferland, Esther Foo, Michael Kotlyar, Brad Holschuh, Maria Gini, Martin Michalowski & Serguei Pakhomov

Authors
  1. Shreya Datar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Libby Ferland
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Esther Foo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Kotlyar
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Brad Holschuh
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Maria Gini
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Martin Michalowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Serguei Pakhomov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Serguei Pakhomov .

Editor information

Editors and Affiliations

  1. Oak-Ridge National Lab (ORNL), Department of Pediatrics, Center for Biomedical Informatics, College of Medicine, The University of Tennessee Health Science Center (UTHSC), Memphis, TN, USA

    Arash Shaban-Nejad

  2. School of Nursing, University of Minnesota, Minneapolis, MN, USA

    Martin Michalowski

  3. IBM Almaden Research Center, San Jos, CA, USA

    Simone Bianco

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Datar, S. et al. (2022). Measuring Physiological Markers of Stress During Conversational Agent Interactions. In: Shaban-Nejad, A., Michalowski, M., Bianco, S. (eds) AI for Disease Surveillance and Pandemic Intelligence. W3PHAI 2021. Studies in Computational Intelligence, vol 1013. Springer, Cham. https://doi.org/10.1007/978-3-030-93080-6_18

Download citation

Publish with us

Policies and ethics

Search

Navigation