Using Perceptual Evaluation to Quantify Cognitive and Visual Driver Distractions

  • Nanxiang Li
  • Carlos BussoEmail author


Developing feedback systems that can detect the attention level of the driver can play a key role in preventing accidents by alerting the driver about possible hazardous situations. Monitoring drivers’ distraction is an important research problem, especially with new forms of technology that are made available to drivers. An important question is how to define reference labels that can be used as ground truth to train machine-learning algorithms to detect distracted drivers. The answer to this question is not simple since drivers are affected by visual, cognitive, auditory, psychological, and physical distractions. This chapter proposes to define reference labels with perceptual evaluations from external evaluators. We describe the consistency and effectiveness of using a visual-cognitive space for subjective evaluations. The analysis shows that this approach captures the multidimensional nature of distractions. The representation also defines natural modes to characterize driving behaviors.


Driver distraction Active safety Driver perception Subjective evaluation Driving performance 



The authors would like to thank Dr. John Hansen for his support with the UTDrive Platform. We want to thank the Machine Perception Lab (MPLab) at The University of California, San Diego, for providing the CERT software. The authors are also thankful to Ms. Rosarita Khadij M Lubag for her support and efforts with the data collection.


  1. 1.
    M. C. F. Aguilo, Development of guidelines for in-vehicle information presentation: text vs. speech, Master’s thesis, Virginia Polytechnic Institute and State University, Blacksburg, Virginia, May 2004Google Scholar
  2. 2.
    P. Angkititrakul, D. Kwak, S. Choi, J. Kim, A. Phucphan, A. Sathyanarayana, and J.H.L. Hansen, Getting start with UTDrive: driver-behavior modeling and assessment of distraction for in-vehicle speech systems, in Interspeech 2007, Antwerp, Belgium, August 2007, pp. 1334–1337Google Scholar
  3. 3.
    A. Azman, Q. Meng, E. Edirisinghe, Non intrusive physiological measurement for driver cognitive distraction detection: Eye and mouth movements. In International Conference on Advanced Computer Theory and Engineering (ICACTE 2010), vol. 3, Chengdu, China, August 2010Google Scholar
  4. 4.
    K. M. Bach, M.G. Jaeger, M.B. Skov, and N.G. Thomassen, Interacting with in-vehicle systems: understanding, measuring, and evaluating attention. In Proceedings of the 23rd British HCI Group Annual Conference on People and Computers: Celebrating People and Technology, Cambridge, United Kingdom, September 2009Google Scholar
  5. 5.
    M.S. Bartlett, G.C. Littlewort, M.G. Frank, C. Lainscsek, I. Fasel, J.R. Movellan, Automatic recognition of facial actions in spontaneous expressions. J. Multimedia 1, 22–35 (2006)Google Scholar
  6. 6.
    C. Busso, J. Jain, Advances in multimodal tracking of driver distraction, in Digital Signal Processing for In-Vehicle Systems and Safety, ed. by J. Hansen, P. Boyraz, K. Takeda, H. Abut (Springer, New York, NY, 2011), pp. 253–270Google Scholar
  7. 7.
    Y. Dong, Z. Hu, K. Uchimura, N. Murayama, Driver inattention monitoring system for intelligent vehicles: a review. IEEE Trans. Intel. Trans. Syst. 12(2), 596–614 (2011)CrossRefGoogle Scholar
  8. 8.
    F.A. Drews, M. Pasupathi, D.L. Strayer, Passenger and cell phone conversations in simulated driving. J. Exp. Psychol. Appl. 14(4), 392–400 (2008)CrossRefGoogle Scholar
  9. 9.
    J. Engström, E. Johansson, J. Östlund, Effects of visual and cognitive load in real and simulated motorway driving. Trans. Res. Part F Traffic Psychol. Behav. 8(2), 97–120 (2005)CrossRefGoogle Scholar
  10. 10.
    T. Ersal, H.J.A. Fuller, O. Tsimhoni, J.L. Stein, H.K. Fathy, Model-based analysis and classification of driver distraction under secondary tasks. IEEE Trans. Intel. Trans. Syst. 11(3), 692–701 (2010)CrossRefGoogle Scholar
  11. 11.
    J.P. Foley, Now you see it, now you dont: visual occlusion as a surrogate distraction measurement technique, in Driver Distraction: Theory Effects, and Mitigation, ed. by M.A. Regan, J.D. Lee, K.L. Young (CRC Press, Boca Raton, FL, 2008), pp. 123–134CrossRefGoogle Scholar
  12. 12.
    A. L. Glaze, J.M. Ellis, Pilot study of distracted drivers. Technical report, Transportation and Safety Training Center, Virginia Commonwealth University, Richmond, VA, USA, January 2003Google Scholar
  13. 13.
    P. Green, The 15-second rule for driver information systems. In Intelligent Transportation Society (ITS) America Ninth Annual Meeting, Washington, DC, USA, April 1999Google Scholar
  14. 14.
    J.L. Harbluk, Y.I. Noy, P.L. Trbovich, M. Eizenman, An on-road assessment of cognitive distraction: impacts on drivers’ visual behavior and braking performance. Accid. Anal. Prev. 39(2), 372–379 (2007)CrossRefGoogle Scholar
  15. 15.
    J. Jain, C. Busso. Analysis of driver behaviors during common tasks using frontal video camera and CAN-Bus information. In IEEE International Conference on Multi media and Expo (ICME 2011), Barcelona, Spain, July 2011Google Scholar
  16. 16.
    J.J. Jain, C. Busso. Assessment of driver’s distraction using perceptual evaluations, self assessments and multimodal feature analysis. In 5th Biennial Workshop on DSP for In-Vehicle Systems, Kiel, Germany, September 2011Google Scholar
  17. 17.
    S.G. Klauer, T.A. Dingus, V.L. Neale, J.D. Sudweeks, D.J. Ramsey, The impact of driver inattention on near-crash/crash risk: an analysis using the 100-car naturalistic driving study data. Technical Report DOT HS 810 594, National Highway Traffic Safety Administration, Blacksburg, VA, USA, April 2006Google Scholar
  18. 18.
    J.D. Lee, B. Caven, S. Haake, T.L. Brown, Speech-based interaction with in-vehicle computers: the effect of speech-based e-mail on drivers’ attention to the road-way. Hum. Factors 43(4), 631–640 (Winter 2001)Google Scholar
  19. 19.
    J.D. Lee, D.V. McGehee, T.L. Brown, M.L. Reyes, Collision warning timing, driver distraction, and driver response to imminent rear-end collisions in a high-fidelity driving simulator. Hum. Factors 44, 314–334 (Summer 2002)Google Scholar
  20. 20.
    Y. Liang, M.L. Reyes, J.D. Lee, Real-time detection of driver cognitive distraction using support vector machines. IEEE Trans. Intel. Trans. Syst. 8(2), 340–350 (2007)CrossRefGoogle Scholar
  21. 21.
    S. Mattes, A. Hallén, Surrogate distraction measurement techniques: the lane change test, in Driver Distraction: Theory, Effects, and Mitigation, ed. by M.A. Regan, J.D. Lee, K.L. Young (CRC Press, Boca Raton, FL, 2008), pp. 107–122CrossRefGoogle Scholar
  22. 22.
    J.C. McCall, M.M. Trivedi, Driver behavior and situation aware brake assistance for intelligent vehicles. Proc. IEEE 95(2), 374–387 (2007)CrossRefGoogle Scholar
  23. 23.
    B. Mehler, B. Reimer, J.F. Coughlin, J.A. Dusek, Impact of incremental increases in cognitive workload on physiological arousal and performance in young adult drivers. Trans. Res. Record 2138, 6–12 (2009)CrossRefGoogle Scholar
  24. 24.
    V. Neale, T. Dingus, S. Klauer, J. Sudweeks, M. Goodman, An overview of the 100-car naturalistic study and findings. Technical Report Paper No. 05-0400, National Highway Traffic Safety Administration, June 2005Google Scholar
  25. 25.
    W. Piechulla, C. Mayser, H. Gehrke, W. Koenig, Reducing drivers’ mental work-load by means of an adaptive man–machine interface. Trans. Res. Part F Traffic Psychol. Behav. 6(4), 233–248 (2003)CrossRefGoogle Scholar
  26. 26.
    F. Putze, J.-P. Jarvis, T. Schultz, Multimodal recognition of cognitive workload for multitasking in the car. In International Conference on Pattern Recognition (ICPR 2010), Istanbul, Turkey, August 2010Google Scholar
  27. 27.
    T.A. Ranney, Driver distraction: a review of the current state-of-knowledge. Technical Report DOT HS 810 787, National Highway Traffic Safety Administration, April 2008Google Scholar
  28. 28.
    T.A. Ranney, W.R. Garrott, M.J. Goodman, NHTSA driver distraction research: past, present, and future. Technical Report Paper No. 2001-06-0177, National High- way Traffic Safety Administration, June 2001Google Scholar
  29. 29.
    E.M. Rantanen, J.H. Goldberg, The effect of mental workload on the visual field size and shape. Ergonomics 42(6), 816–834 (1999)CrossRefGoogle Scholar
  30. 30.
    M.A. Recarte, L.M. Nunes, Mental workload while driving: effects on visual search, discrimination, and decision making. J. Exp. Psychol. Appl. 9(2), 119–137 (2003)CrossRefGoogle Scholar
  31. 31.
    A. Sathyanarayana, S. Nageswaren, H. Ghasemzadeh, R. Jafari, J.H.L. Hansen, Body sensor networks for driver distraction identification. In IEEE International Conference on Vehicular Electronics and Safety (ICVES 2008), Columbus, OH, USA, September 2008Google Scholar
  32. 32.
    D.L. Strayer, J.M. Cooper, F.A. Drews. What do drivers fail to see when conversing on a cell phone? In Proceedings of Human Factors and Ergonomics Society Annual Meeting, volume 48, New Orleans, LA, USA, September 2004Google Scholar
  33. 33.
    D.L. Strayer, J.M. Watson, F.A. Drews, Cognitive distraction while multitasking in the automobile, in The Psychology of Learning and Motivation, ed. by B.H. Ross, vol. 54 (Academic, Burlington, MA, 2011), pp. 29–58Google Scholar
  34. 34.
    J.C. Stutts, D.W. Reinfurt, L. Staplin, E.A. Rodgman, The role of driver distraction in traffic crashes. Technical report, AAA Foundation for Traffic Safety, Washington, DC, USA, May 2001Google Scholar
  35. 35.
    F. Tango, M. Botta, Evaluation of distraction in a driver-vehicle-environment framework: an application of different data-mining techniques, in Advances in Data Mining. Applications and Theoretical Aspects, ed. by P. Perner. Lecture Notes in Computer Science, vol. 5633 (Springer, Berlin, 2009), pp. 176–190CrossRefGoogle Scholar
  36. 36.
    T.W. Victor, J. Engstroem, J.L. Harbluk, Distraction assessment methods based on visual behavior and event detection, in Driver Distraction: Theory, Effects, and Mitigation, ed. by M.A. Regan, J.D. Lee, K.L. Young (CRC Press, Boca Raton, FL, 2008), pp. 135–165CrossRefGoogle Scholar
  37. 37.
    W. Wierwille, L. Tijerina, S. Kiger, T. Rockwell, E. Lauber, A. Bittner Jr, Final report supplement—task 4: review of workload and related research. Technical Report DOT HS 808 467, U.S. Department of Transportation, National Highway Traffic Safety Administration, Washington, DC, USA, October 1996Google Scholar
  38. 38.
    Q. Wu, An overview of driving distraction measure methods. In IEEE 10th International Conference on Computer-Aided Industrial Design & Conceptual Design (CAID CD 2009), Wenzhou, China, November 2009Google Scholar
  39. 39.
    K.L. Young, M.A. Regan, J.D. Lee, Measuring the effects of driver distraction: direct driving performance methods and measures, in Driver Distraction: Theory Effects, and Mitigation, ed. by M.A. Regan, J.D. Lee, K.L. Young (CRC Press, Boca Raton, FL, 2008), pp. 85–105CrossRefGoogle Scholar
  40. 40.
    Y. Zhang, Y. Owechko, J. Zhang. Driver cognitive workload estimation: a data-driven perspective. In IEEE Intelligent Transportation Systems, Washington, DC, USA, October 2004, pp. 642–647Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.The University of Texas at DallasRichardsonUSA

Personalised recommendations