Skip to main content
Book cover

International Conference on Human-Computer Interaction

HCII 2022: HCI in Mobility, Transport, and Automotive Systems pp 218–230Cite as

Understanding Drivers’ Physiological Responses in Different Road Conditions

  • 379 Accesses

Part of the Lecture Notes in Computer Science book series (LNCS,volume 13335)

Abstract

Although the driver’s emotion has been studied in the different driving environments (such as city and highway), understanding what eye metrics and facial expressions correspond to specific emotion and behavior based on subjective and biosensor data to study emotion in depth is not well researched in previous studies. Using an eye-integrated human-in-the-loop (HTIL) simulation experiment, we studied how drivers’ facial expressions and ocular measurements relate to emotions. We found that the driving environment could significantly affect drivers’ emotions, which is evident in their facial expressions and eye metrics data. In addition, such outcomes provide knowledge to human-computer-interaction (HCI) practitioners on designing emotion recognition systems in cars to have a robust understanding of the drivers’ emotions and help progress multimodal emotion recognition.

Keywords

  • Psychological conditions
  • Driving behavior
  • Eye tracking
  • Facial expression

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-031-04987-3_15
  • Chapter length: 13 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-031-04987-3
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.

References

  1. Tavakoli, A., Balali, V., Heydarian, A.: A Multimodal Approach for Monitoring Driving Behavior and Emotions (2020)

    Google Scholar 

  2. Kleinginna, P.R., Kleinginna, A.M.: A categorized list of motivation definitions, with a suggestion for a consensual definition. Motiv. Emot. 5(3), 263–291 (1981). https://doi.org/10.1007/BF00993889

    CrossRef  Google Scholar 

  3. Pittermann, J., Pittermann, A., Minker, W.: Emotion recognition and adaptation in spoken dialogue systems. Int. J. Speech Technol. 13(1), 49–60 (2010). https://doi.org/10.1007/s10772-010-9068-y

    CrossRef  Google Scholar 

  4. Leahu, L., Schwenk, S., Sengers, P.: Subjective objectivity: negotiating emotional meaning. In: Proceedings of the 7th ACM Conference on Designing Interactive Systems, pp. 425–434 (2008)

    Google Scholar 

  5. Picard, R.W., Klein, J.: Computers that recognise and respond to user emotion: theoretical and practical implications. Interact. Comput. 14(2), 141–169 (2002)

    CrossRef  Google Scholar 

  6. Cohn, J.F.: Foundations of human computing: facial expression and emotion. In: Proceedings of the 8th International Conference on Multimodal Interfaces, pp. 233–238 (2006)

    Google Scholar 

  7. Martis, J.E.: Effective emotion recognition of expressions from facial features. 5(06), 4–7 (2017)

    Google Scholar 

  8. Geiger, A., Brandenburg, E., Stark, R.: Natural virtual reality user interface to define assembly sequences for digital human models. Appl. Syst. Innov. 3(1), 15 (2020)

    CrossRef  Google Scholar 

  9. Sedenberg, E., Wong, R., Chuang, J.: A window into the soul: biosensing in public. arXiv preprint arXiv:1702.04235 (2017)

  10. Liu, L., et al.: Deep learning for generic object detection: a survey. arXiv 2018. arXiv preprint arXiv:1809.02165 (2019)

  11. Martinez-Conde, S., Macknik, S.L., Hubel, D.H.: The role of fixational eye movements in visual perception. Nat. Rev. Neurosci. 5(3), 229–240 (2004)

    CrossRef  Google Scholar 

  12. Baujon, J., Basset, M., Gissinger, G.L.: Visual behaviour analysis and driver cognitive model. In: Proceedings of the 3rd IFAC Workshop on Advances in Automotive Control, Karlsruhe, Germany, pp. 47–52 (2001)

    Google Scholar 

  13. Brackstone, M., Waterson, B.: Are we looking where we are going? An exploratory examination of eye movement in high-speed driving. In: Proceedings of the 83rd Transportation Research Board Annual Meeting, vol. 2, p. 602 (2004)

    Google Scholar 

  14. Yan, Y., Yuan, H., Wang, X., Xu, T., Liu, H.: Study on driver’s fixation variation at entrance and inside sections of tunnel on highway. Adv. Mech. Eng. 7(1), 273427 (2015)

    CrossRef  Google Scholar 

  15. Nadal, M., Munar, E., Marty, G., Cela-Conde, C.J.: Visual complexity and beauty appreciation: explaining the divergence of results. Empirical Stud. Arts 28(2), 173–191 (2010)

    CrossRef  Google Scholar 

  16. Chan, M., Singhal, A.: Emotion matters: Implications for distracted driving. Saf. Sci. 72, 302–309 (2015)

    CrossRef  Google Scholar 

  17. Zhang, W., Zhang, X., Feng, Z., Liu, J., Zhou, M., Wang, K.: The fitness-to-drive of shift-work taxi drivers with obstructive sleep apnea: an investigation of self-reported driver behavior and skill. Transp. Res. Part F: Traffic Psychol. Behav. 59, 545–554 (2018)

    CrossRef  Google Scholar 

  18. Eherenfreund-Hager, A., Taubman-Ben-Ari, O., Toledo, T., Farah, H.: The effect of positive and negative emotions on young drivers a simulator study. Transp. Res. Part F: Traffic Psychol. Behav. 49, 236–243 (2017)

    Google Scholar 

  19. Du, N., et al.: Examining the effects of emotional valence and arousal on takeover performance in conditionally automated driving. Transp. Res. Part C: Emerg. Technol. 112, 78–87 (2020)

    CrossRef  Google Scholar 

  20. Hedlund, J., Simpson, H.M., Mayhew, D.R.: Summary of proceedings and recommendations. In: International Conference on Distracted Driving. The Traffic Injury Research Foundation, The Canadian Automobile Association, Ottawa (2006)

    Google Scholar 

  21. Nyström, M., Holmqvist, K.: An adaptive algorithm for fixation, saccade, and glissade detection in eyetracking data. Behav. Res. Methods 42(1), 188–204 (2010). https://doi.org/10.3758/BRM.42.1.188

    CrossRef  Google Scholar 

  22. Ekman, P., Friesen, W.V.: Facial Action Coding System: Investigator’s Guide. Consulting Psychologists Press, Palo Alto (1978)

    Google Scholar 

  23. Steyer, R., Schwenkmezger, P., Notz, P., Eid, M.: MDMQ questionnaire (English version of MDBF). Jena: Friedrich-Schiller-Universität Jena, Institut für Psychologie, Lehrstuhl für Methodenlehre und Evaluationsforschung (2014). https://www.metheval.uni-jena.de/mdbf.php. Accessed 4 Apr 2016

  24. Meinlschmidt, G., et al.: Smartphone-based psychotherapeutic micro-interventions to improve mood in a real-world setting. Front. Psychol. 7, 1112 (2016)

    CrossRef  Google Scholar 

  25. Hinz, A., Daig, I., Petrowski, K., Brähler, E.: Die stimmung in der deutschen bevölkerung: referenzwerte für den mehrdimensionalen befindlichkeitsfragebogen MDBF. PPmP-Psychother. Psychosom. Med. Psychol. 62(02), 52–57 (2012)

    CrossRef  Google Scholar 

  26. Park, J., Abdel-Aty, M., Yina, W., Mattei, I.: Enhancing in-vehicle driving assistance information under connected vehicle environment. IEEE Trans. Intell. Transp. Syst. 20(9), 3558–3567 (2018)

    CrossRef  Google Scholar 

  27. Lynch, B.K.: Designing qualitative research by catherine marshall an Gretchen B. Rossman. Issues Appl. Linguist. 1(2), 1–9 (1990)

    CrossRef  Google Scholar 

  28. Schutte, N.S., Malouff, J.M., Thorsteinsson, E.B., Bhullar, N., Rooke, S.E.: A meta-analytic investigation of the relationship between emotional intelligence and health. Pers. Individ. Differ. 42(6), 921–933 (2007)

    CrossRef  Google Scholar 

  29. Guarnera, M., Hichy, Z., Cascio, M.I., Carrubba, S.: Facial expressions and ability to recognize emotions from eyes or mouth in children. Eur. J. Psychol. 11(2), 183 (2015)

    CrossRef  Google Scholar 

  30. Li, J., Jin, K., Zhou, D., Kubota, N., Zhaojie, J.: Attention mechanism-based CNN for facial expression recognition. Neurocomputing 411, 340–350 (2020)

    CrossRef  Google Scholar 

  31. Hassib, M., Braun, M., Pfleging, B., Alt, F.: Detecting and influencing driver emotions using psycho-physiological sensors and ambient light. In: Lamas, D., Loizides, F., Nacke, L., Petrie, H., Winckler, M., Zaphiris, P. (eds.) Human-Computer Interaction – INTERACT 2019, vol. 11746, pp. 721–742. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-29381-9_43

    CrossRef  Google Scholar 

  32. Mesken, J.: Determinants and consequences of drivers’ emotions. Stichting Wetenschappelijk Onderzoek Verkeersveiligheid SWOV (2006)

    Google Scholar 

  33. Remington, R.W.: Attention and saccadic eye movements. J. Exp. Psychol.: Hum. Percept. Perform. 6(4), 726 (1980)

    Google Scholar 

  34. Murphy-Chutorian, E., Trivedi, M.M.: Head pose estimation and augmented reality tracking: an integrated system and evaluation for monitoring driver awareness. IEEE Trans. Intell. Transp. Syst. 11(2), 300–311 (2010)

    CrossRef  Google Scholar 

  35. Sahayadhas, A., Sundaraj, K., Murugappan, M.: Detecting driver drowsiness based on sensors: a review. Sensors 12(12), 16937–16953 (2012)

    CrossRef  Google Scholar 

  36. Mashko, A.: Subjective methods for assessment of driver drowsiness. Acta Polytech. CTU Proc. 12, 64–67 (2017)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Industrial and Manufacturing Systems Engineering Department, University of Missouri, Columbia, MO, USA

    Sara Mostowfi & Jung Hyup Kim

Authors
  1. Sara Mostowfi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jung Hyup Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jung Hyup Kim .

Editor information

Editors and Affiliations

  1. Technische Universität Ilmenau, Ilmenau, Germany

    Prof. Dr. Heidi Krömker

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Mostowfi, S., Kim, J.H. (2022). Understanding Drivers’ Physiological Responses in Different Road Conditions. In: Krömker, H. (eds) HCI in Mobility, Transport, and Automotive Systems. HCII 2022. Lecture Notes in Computer Science, vol 13335. Springer, Cham. https://doi.org/10.1007/978-3-031-04987-3_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-04987-3_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-04986-6

  • Online ISBN: 978-3-031-04987-3

  • eBook Packages: Computer ScienceComputer Science (R0)