On the Necessity and Feasibility of Detecting a Driver’s Emotional State While Driving

  • Michael Grimm
  • Kristian Kroschel
  • Helen Harris
  • Clifford Nass
  • Björn Schuller
  • Gerhard Rigoll
  • Tobias Moosmayr
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4738)


This paper brings together two important aspects of the human-machine interaction in cars: the psychological aspect and the engineering aspect. The psychologically motivated part of this study addresses questions such as why it is important to automatically assess the driver’s affective state, which states are important and how a machine’s response should look like. The engineering part studies how the emotional state of a driver can be estimated by extracting acoustic features from the speech signal and mapping them to an emotion state in a multidimensional, continuous-valued emotion space. Such a feasibility study is performed in an experiment in which spontaneous, authentic emotional utterances are superimposed by car noise of several car types and various road surfaces.


Speech Signal Support Vector Regression Emotion Recognition Road Surface Automatic Speech Recognition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hitzenberger, L.: Man Machine Interaction in Car Information Systems. In: Proceedings of the First International Conference on Language Resources and Evaluation, Granada, pp. 179–182 (1998)Google Scholar
  2. 2.
    Jonsson, I.M., Nass, C., Harris, H., Takayama, L.: Matching In-Car Voice with Driver State: Impact on Attitude and Driving Performance. In: Proceedings of the Third International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design, pp. 173–181 (2005)Google Scholar
  3. 3.
    Nass, C., Jonsson, I.M., Harris, H., Reaves, B., Endo, J., Brave, S., Takayama, L.: Improving Automotive Safety by Pairing Driver Emotion and Car Voice Emotion. In: Proc. CHI (2005)Google Scholar
  4. 4.
    Jones, C., Jonsson, I.M.: Automatic recognition of affective cues in the speech of car drivers to allow appropriate responses. In: Proc. OZCHI (2005)Google Scholar
  5. 5.
    Schuller, B., Lang, M., Rigoll, G.: Recognition of Spontaneous Emotions by Speech within Automotive Environment. In: Proc. 32. Deutsche Jahrestagung für Akustik (DAGA), Braunschweig, Germany, pp. 57–58 (2006)Google Scholar
  6. 6.
    Schuller, B., Arsic, D., Wallhoff, F., Rigoll, G.: Emotion Recognition in the Noise Applying Large Acoustic Feature Sets. In: Proc. Speech Prosody, Dresden, Germany (2006)Google Scholar
  7. 7.
    Csikszentmihalyi, M.: Flow: The Psychology of Optimal Experience. Harper & Row, New York (1991)Google Scholar
  8. 8.
    Hirt, E., Melton, R., McDonald, H., Harackiewicz, J.: Processing goals, task interest, and the mood-performance relationship: A mediational analysis. Journal of Personality and Social Psychology 71, 245–261 (1996)CrossRefGoogle Scholar
  9. 9.
    Groeger, J.: Understanding Driving: Applying Cognitive Psychology to a Complex Everyday Task. Psychology Press, Philadelphia, PA (2000)Google Scholar
  10. 10.
    Isen, A., Rosenzweig, A., Young, M.: The influence of positive affect on clinical problem solving. Medical Decision Making 11(3), 221–227 (1991)CrossRefGoogle Scholar
  11. 11.
    Galovski, T., Blanchard, E.: Road rage: A domain for psychological intervention? Aggression and Violent Behavior 9(2), 105–127 (2004)CrossRefGoogle Scholar
  12. 12.
    Gross, J., Levenson, R.: Emotion elicitation using films. Cognition & Emotion 9, 87–108 (1995)CrossRefGoogle Scholar
  13. 13.
    Detenber, B., Reeves, B.: A bio-informational theory of emotion: Motion and image size effects on viewers. Journal of Communication 46(3), 66–84 (1996)CrossRefGoogle Scholar
  14. 14.
    Izard, C.: Patterns of Emotions. Academic Press, New York (1972)Google Scholar
  15. 15.
    Nass, C., Gong, L.: Social aspects of speech interfaces from an evolutionary perspective: Experimental research and design implications. Communications of the ACM 43, 36–43 (2000)CrossRefGoogle Scholar
  16. 16.
    Nass, C., Brave, S.: Wired for Speech: How Voice Activates and Enhances the Human-Computer Relationship. MIT Press, Cambridge, MA (2005)Google Scholar
  17. 17.
    Grimm, M., Kroschel, K.: Rule-based emotion classification using acoustic features. In: Proc. 3rd International Conference on Telemedicine and Multimedia Communication (ICTMC) (2005)Google Scholar
  18. 18.
    Lang, P.: The Cognitive Psychophysiology of Emotion: Anxiety and the Anxiety Disorders. Lawrence Erlbaum, Hillsdale, NJ (1985)Google Scholar
  19. 19.
    Grimm, M., Kroschel, K.: Evaluation of natural emotions using self assessment manikins. In: Proc. ASRU, pp. 381–385 (2005)Google Scholar
  20. 20.
    Grimm, M., Mower, E., Kroschel, K., Narayanan, S.: Primitives-based evaluation and estimation of emotions in speech. Speech Communication (Journal accepted for publication, 2007)Google Scholar
  21. 21.
    Setiawan, P., Suhadi, S., Fingscheidt, T., Stan, S.: Robust Speech Recognition for Mobil Devices in Car Noise. In: Proc. Interspeech, Lisbon, Portugal (2005)Google Scholar
  22. 22.
    Grimm, M., Kroschel, K., Narayanan, S.: Support vector regression for automatic recognition of spontaneous emotions in speech. In: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Proccessing (ICASSP) (accepted for Publication, 2007)Google Scholar
  23. 23.
    Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., Taylor, J.: Emotion recognition in human-computer interaction. IEEE Signal Processing Magazine 18(1), 32–80 (2001)CrossRefGoogle Scholar
  24. 24.
    Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)zbMATHGoogle Scholar
  25. 25.
    Smola, A., Schölkopf, B.: A tutorial on support vector regression. Technical report, NeuroCOLT2 (1998)Google Scholar
  26. 26.
    Kammeyer, K.D., Kroschel, K.: Digitale Signalverarbeitung, 4th edn., Teubner Stuttgart (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Michael Grimm
    • 1
  • Kristian Kroschel
    • 1
  • Helen Harris
    • 2
  • Clifford Nass
    • 2
  • Björn Schuller
    • 3
  • Gerhard Rigoll
    • 3
  • Tobias Moosmayr
    • 4
  1. 1.Universität Karlsruhe (TH), Institut für Nachrichtentechnik, 76128 KarlsruheGermany
  2. 2.Stanford University, Department of Communication, Stanford, CA 94305-2050USA
  3. 3.Technische Universität München, Institute for Human-Machine Communication, 80290 MünchenGermany
  4. 4.BMW Group, Forschungs- und Innovationszentrum, Akustik, Komfort und Werterhaltung, 80788 MünchenGermany

Personalised recommendations