Skip to main content
SpringerLink
Log in

Advances in Multimodal Tracking of Driver Distraction

  • Chapter
  • First Online:
Digital Signal Processing for In-Vehicle Systems and Safety

Abstract

This chapter discusses research efforts focused on tracking driver distraction using multimodal features. A car equipped with various sensors is used to collect a database with real driving conditions. During the recording, the drivers were asked to perform common secondary tasks such as operating a cell phone, talking to another passenger, and changing the radio stations. We analyzed the differences observed across multimodal features when the driver was engaged in these secondary tasks. The study considers features extracted from the controller area network bus (CAN-bus), a frontal camera facing the driver, and a microphone. These features are used to predict the distraction level of the drivers. The output of the proposed regression model has high correlation with human subjective evaluations (ρ = 0.728), which validates our approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. TA Ranney, WR Garrott, MJ Goodman (2001) NHTSA driver distraction research: past, present, and future. Technical Report Paper No. 2001-06-0177, National Highway Traffic Safety Administration, June 2001

    Google Scholar 

  2. V Neale, T Dingus, S Klauer, J Sudweeks, M Goodman (2005) An overview of the 100-car naturalistic study and findings. Technical Report Paper No. 05-0400, National Highway Traffic Safety Administration, June 2005

    Google Scholar 

  3. TA Ranney (2008) Driver distraction: a review of the current state-of-knowledge. Technical Report DOT HS 810 787, National Highway Traffic Safety Administration, April 2008

    Google Scholar 

  4. I Trezise, EG Stoney, B Bishop, J Eren, A Harkness, C Langdon, T Mulder (2006) Inquiry into driver distraction: report of the road safety committee on the inquiry into driver distraction. Technical Report No. 209 Session 2003–2006, Melbourne: Road Safety Committee, Parliament of Victoria, August 2006

    Google Scholar 

  5. J Jain, C Busso (2011) Analysis of driver behaviors during common tasks using frontal video camera and CAN-Bus information. In: IEEE international conference on multimedia and expo (ICME 2011), Barcelona, July 2011

    Google Scholar 

  6. JJ Jain, C Busso (2011) Assessment of driver’s distraction using perceptual evaluations, self assessments and multimodal feature analysis. In: 5th Biennial workshop on DSP for in-vehicle systems, Kiel, September 2011

    Google Scholar 

  7. P Angkititrakul, D Kwak, S Choi, J Kim, A Phucphan, A Sathyanarayana, JHL Hansen (2007) Getting start with UTDrive: driver-behavior modeling and assessment of distraction for in-vehicle speech systems. In: Interspeech 2007, Antwerp, August 2007, pp 1334–1337

    Google Scholar 

  8. Lin C-T, Wu R-C, Liang S-F, Chao W-H, Chen Y-J, Jung T-P (2005) EEG-based drowsiness estimation for safety driving using independent component analysis. IEEE Trans Circ Syst I: Regular Papers 52(12):2726–2738

    Article  Google Scholar 

  9. T Rahman, S Mariooryad, S Keshavamurthy, G Liu, JHL Hansen, C Busso (2011) Detecting sleepiness by fusing classifiers trained with novel acoustic features. In: 12th annual conference of the international speech communication association (Interspeech’2011), Florence, August 2011

    Google Scholar 

  10. P Angkititrakul, M Petracca, A Sathyanarayana, JHL Hansen (2007) UTDrive: driver behavior and speech interactive systems for in-vehicle environments. In: IEEE intelligent vehicles symposium, Istanbul, June 2007, pp 566–569

    Google Scholar 

  11. M Kutila, M Jokela, G Markkula, MR Rue (2007) Driver distraction detection with a camera vision system. In: IEEE international conference on image processing (ICIP 2007), vol 6. San Antonio, September 2007, pp 201–204

    Google Scholar 

  12. Y Dong, Z Hu, K Uchimura, N Murayama (2009) Driver inattention monitoring system for intelligent vehicles: a review. In: IEEE intelligent vehicles symposium, Xi’an, June 2009, pp 875–880

    Google Scholar 

  13. Liang Y, Reyes ML, Lee JD (2007) Real-time detection of driver cognitive distraction using support vector machines. IEEE Trans Intell Transport Syst 8(2):340–350

    Article  Google Scholar 

  14. MC Su, CY Hsiung, DY Huang (2006) A simple approach to implementing a system for monitoring driver inattention. In: IEEE international conference on systems, man and cybernetics (SMC 2006), vol 1. Taipei, October 2006, pp 429–433

    Google Scholar 

  15. Damousis IG, Tzovaras D (2008) Fuzzy fusion of eyelid activity indicators for hypovigilance-related accident prediction. IEEE Trans Intell Transportation Syst 9(3):491–500

    Article  Google Scholar 

  16. F Putze, J-P Jarvis, T Schultz (2010) Multimodal recognition of cognitive workload for multitasking in the car. In: International conference on pattern recognition (ICPR 2010), Istanbul, August 2010

    Google Scholar 

  17. Bergasa LM, Nuevo J, Sotelo MA, Barea R, Lopez ME (2006) Real-time system for monitoring driver vigilance. IEEE Trans Intell Transport Syst 7(1):63–77

    Article  Google Scholar 

  18. Ersal T, Fuller HJA, Tsimhoni O, Stein JL, Fathy HK (2010) Model-based analysis and classification of driver distraction under secondary tasks. IEEE Trans Intell Transport Syst 11(3):692–701

    Article  Google Scholar 

  19. Tango F, Botta M (2009) Evaluation of distraction in a driver-vehicle-environment framework: an application of different data-mining techniques. In: Perner P (ed) Advances in data mining. Applications and theoretical aspects, volume 5633 of lecture notes in computer science. Springer, Berlin/Heidelberg, pp 176–190

    Chapter  Google Scholar 

  20. KM Bach, MG Jaeger, MB Skov, NG Thomassen (2009) Interacting with in-vehicle systems: understanding, measuring, and evaluating attention. In: Proceedings of the 23rd British HCI Group annual conference on people and computers: celebrating people and technology, Cambridge, UK, September 2009

    Google Scholar 

  21. A Sathyanarayana, P Boyraz, Z Purohit, R Lubag, JHL Hansen (2010) Driver adaptive and context aware active safety systems using CAN-bus signals. In: IEEE intelligent vehicles symposium (IV 2010), San Diego, June 2010

    Google Scholar 

  22. A Sathyanarayana, S Nageswaren, H Ghasemzadeh, R Jafari, JHL Hansen (2008) Body sensor networks for driver distraction identification. In: IEEE international conference on vehicular electronics and safety (ICVES 2008), Columbus, September 2008

    Google Scholar 

  23. J Yang, TN Chang, E Hou (2010) Driver distraction detection for vehicular monitoring. In: Annual conference of the IEEE industrial electronics society (IECON 2010), Glendale, November 2010

    Google Scholar 

  24. Murphy-Chutorian E, Trivedi MM (2010) Head pose estimation and augmented reality tracking: an integrated system and evaluation for monitoring driver awareness. IEEE Trans Intell Transport Syst 11(2):300–311

    Article  Google Scholar 

  25. Harbluk JL, Noy YI, Trbovich PL, Eizenman M (2007) An on-road assessment of cognitive distraction: impacts on drivers’ visual behavior and braking performance. Accid Anal Prev 39(2):372–379

    Article  Google Scholar 

  26. Perez A, Garcia MI, Nieto M, Pedraza JL, Rodriguez S, Zamorano J (2010) Argos: an advanced in-vehicle data recorder on a massively sensorized vehicle for car driver behavior experimentation. IEEE Trans Intell Transport Syst 11(2):463–473

    Article  Google Scholar 

  27. Abut H, Erdoğan H, Ercil A, Curuklu B, Koman HC, Taş F, Argunşah AO, Coşar S, Akan B, Karabalkan H et al (2009) Real-world data collection with “UYANIK”. In: Takeda K, Erdoğan H, Hansen JHL, Abut H (eds) In-vehicle corpus and signal processing for driver behavior. Springer, New York, pp 23–43

    Chapter  Google Scholar 

  28. P Boyraz, X Yang, JHL Hansen (2009) Computer vision applications for context-aware intelligent vehicles. In: 4th Biennial workshop on DSP for in-vehicle systems and safety, Dallas, June 2009

    Google Scholar 

  29. DL Strayer, JM Cooper, FA Drews (2004) What do drivers fail to see when conversing on a cell phone? In: Proceedings of human factors and ergonomics society annual meeting, vol 48. New Orleans, September 2004

    Google Scholar 

  30. MS Bartlett, GC Littlewort, MG Frank, C Lainscsek, I Fasel, JR Movellan (2006) Automatic recognition of facial actions in spontaneous expressions. Journal of Multimedia 6(1):22–35, September 2006

    Google Scholar 

  31. Mendenhall W, Sincich T (2006) Statistics for engineering and the sciences. Prentice-Hall, Upper Saddle River

    Google Scholar 

  32. Berka C, Levendowski DJ, Lumicao MN, Yau A, Davis G, Zivkovic VT, Olmstead RE, Tremoulet PD, Craven PL (2007) EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviat Space Environ Med 78(5):231–244

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The University of Texas at Dallas, Richardson, TX, 75080, USA

    Carlos Busso & Jinesh Jain

Authors
  1. Carlos Busso
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinesh Jain
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carlos Busso .

Editor information

Editors and Affiliations

  1. , Electrical Engineering, The University of Texas at Dallas, W Campbell Rd 800, Richardson, 75080, USA

    John H.L. Hansen

  2. , ITU Mechanical Engineering, İTÜ Makina Fakültesi, İnönü Caddesi 65, Istanbul, 34437, Turkey

    Pinar Boyraz

  3. Nagoya University, Nagoya, 464-8603, Japan

    Kazuya Takeda

  4. Cummins Place 9985, San Diego, 92131, USA

    Hüseyin Abut

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Busso, C., Jain, J. (2012). Advances in Multimodal Tracking of Driver Distraction. In: Hansen, J., Boyraz, P., Takeda, K., Abut, H. (eds) Digital Signal Processing for In-Vehicle Systems and Safety. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-9607-7_18

Download citation

  • DOI: https://doi.org/10.1007/978-1-4419-9607-7_18

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4419-9606-0

  • Online ISBN: 978-1-4419-9607-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation