Learning-Based Driver Workload Estimation

  • Yilu Zhang
  • Yuri Owechko
  • Jing Zhang
Part of the Studies in Computational Intelligence book series (SCI, volume 132)

A popular definition of workload is given by O’Donnell and Eggmeir, which states that “The term workload refers to that portion of the operator’s limited capacity actually required to perform a particular task” [1]. In the vehicle environment, the “particular task” refers to both the vehicle control, which is the primary task, and other secondary activities such as listening to the radio. Three major types of driver workload are usually studied, namely, visual, manual, and cognitive. Auditory workload is not treated as a major type of workload in the driving context because the auditory perception is not considered as a major requirement to perform a driving task. Even when there is an activity that involves audition, the driver is mostly affected cognitively.

Lately, the advanced computer and telecommunication technology is introducing many new in-vehicle information systems (IVISs), which give drivers more convenient and pleasant driving experiences. Active research is being conducted to provide IVISs with both high functionality and high usability. On the usability side, driver's workload is a heated topic advancing in at least two major directions. One is the offline assessment of the workload imposed by IVISs, which can be used to improve the design of IVISs. The other effort is the online workload estimation, based on which IVISs can provide appropriate service at appropriate time, which is usually termed as Workload Management. For example, the incoming phone call may be delayed if the driver is engaged in a demanding maneuver.

Among the three major types of driver workload, cognitive workload is the most difficult to measure. For example, withdrawing hands from the steering wheel to reach for a coffee cup requires extra manual workload. It also may require extra visual workload in that the position of the cup may need to be located. Both types of workload are directly measurable through such observations as hands-off-wheel and eyes-off- road time. On the other hand, engaging in thinking (the so-called minds-off-road phenomenon) is difficult to detect. Since the cognitive workload level is internal to the driver, it can only be inferred based on the information that is observable. In this chapter, we report some of our research results on driver's cognitive workload estimation.1 After the discussion of the existing practices, we propose a new methodology to design driver workload estimation systems, that is, using machine-learning techniques to derive optimized models to index workload. The advantage of this methodology will be discussed, followed by the presentation of some experimental results. This chapter concludes with discussion of future work.


Secondary Task Vehicle Speed Pupil Diameter Intelligent Transportation System Cognitive Workload 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    R.D. O’Donnel and F.T. Eggemeier, “Workload assessment methodology,” in Handbook of Perception and Human Performance, Vol II, Cognitive Processes and Performance, K.R. Boff, L. Kaufman, and J.P. Thomas, Eds. Wiley, New York, 1986.Google Scholar
  2. 2.
    C.D. Wickens and J.G. Hollands, Engineering Psychology and Human Performance, Prentice-Hall, Upper Saddle River, NJ, 3rd edition, 2000.Google Scholar
  3. 3.
    S.G. Hart and L.E. Seaveland, “Development of NASA-TLS (task load index): results of empirical and theoretical research,” in Human Mental Workload, P.A. Hancock and N. Meshkati, Eds., North Holland, Amsterdam, 1988.Google Scholar
  4. 4.
    G.B. Reid and T.E. Nygren, “The subjective workload assessment technique: a scaling procedure for measuring mental workload,” in Human Mental Workload, P.A. Hancock and N. Meshkati, Eds., North Holland, Amsterdam, 1988.Google Scholar
  5. 5.
    L.S. Angell, R.A. Young, J.M. Hankey, and T.A. Dingus, “An evaluation of alternative methods for assessing driver workload in the early development of in-vehicle information systems,” in Proceedings of SAE Government/Industry Meeting, Washington, DC, May 2002.Google Scholar
  6. 6.
    J. Hurwitz and D.J. Wheatley, “Using driver performance measures to estimate workload,” in Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting, Santa Monica, CA, pp. 1804–1808, 2002.Google Scholar
  7. 7.
    C. Mayser, W. Piechulla, K.-E. Weiss, and W. König, “Driver workload monitoring,” in Proceedings of the Internationale Ergonomie-Konferenz der GfA, ISOES und FEES, Mnchen, Germany, May 7–9, 2003.Google Scholar
  8. 8.
    O. Nakayama, T. Futami, T. Nakamura, and E.R. Boer, “Development of a steering entropy method for evaluating driver workload,” in Proceedings of SAE International Congress and Exposition Detroit, Detroit, MI, March 1999.Google Scholar
  9. 9.
    C.D. Wickens, J. Helleberg, J. Goh, X. Xu, and B. Horrey, “Pilot task management: testing an attentional expected value model of visual scanning,” Tech. Rep. ARL-01-14/NASA-01-7, University of Illinois, Aviation Research Lab, Savoy, IL, 2001.Google Scholar
  10. 10.
    S.K. Card, T.P. Moran, and A. Newell, “The model human processor: an engineering model of human performance,” in Handbook of Perception and Human Performance, Wiley, New York, pp. 1–35, 1986.Google Scholar
  11. 11.
    D.E. Kieras and D.E. Meyer, “An overview of the epic architecture for cognition and performance with application to human-computer interaction,” Human-Computer Interaction, 12, 391–438, 1997.CrossRefGoogle Scholar
  12. 12.
    J.R. Anderson and C. Lebiere, The Atomic Components of Thought, Erlbaum, Mahwah, NJ, 1998.Google Scholar
  13. 13.
    M.A. Recarte and L.M. Nunes, “Effects of verbal and spatial-imagery task on eye fixations while driving,” Journal of Experimental Psychology: Applied, 6(1), 31–43, 2000.CrossRefGoogle Scholar
  14. 14.
    J.L. Harbluk and Y.I. Noy, “The impact of cognitive distraction on driver visual behaviour and vehicle control,” Technical Report, Transport Canada,, February 2002.
  15. 15.
    M. Rahimi, R.P. Briggs, and D.R. Thom, “A field evaluation of driver eye and head movement strategies toward environmental targets and distracters,” Applied Ergonomics, 21(4), 267–274, 1990.CrossRefGoogle Scholar
  16. 16.
    P.R. Chapman and G. Underwood, “Visual search of dynamic scenes: event types and the role of experience in viewing driving situations,” in Eye Guidance in Reading and Science Perception, G. Underwood, Ed., Elsevier, Amsterdam, pp. 369–393, 1998.CrossRefGoogle Scholar
  17. 17.
    Y. Zhang, Y. Owechko, and J. Zhang, “Machine learning-based driver workload estimation,” in Proceedings of the 7th International Symposium on Advanced Vehicle Control, Arnhem, Netherlands, August 23–27, 2004.Google Scholar
  18. 18.
    Y. Zhang, Y. Owechko, and J. Zhang, “Driver cognitive workload estimation: a data-driven perspective,” in Proceedings of 7th International IEEE Conference on Intelligent Transportation Systems, Washington D.C., October 3–6, 2004.Google Scholar
  19. 19.
    T.M. Mitchell, Machine Learning, WCB/McGraw-Hill, Boston, MA, 1997.zbMATHGoogle Scholar
  20. 20.
    P. Bhagat, Pattern Recognition in Industry, Elsevier Science, Oxford, UK, 2005.Google Scholar
  21. 21.
    K.-F. Kraiss, “Implementation of user-adaptive assistants with neural operator models,” Control Engineering Practice, 3(2), 249–256, 1995.CrossRefGoogle Scholar
  22. 22.
    J. Forbes, T. Huang, K. Kanazawa, and S. Russell, “Batmobile: Towards a bayesian automated taxi,” in Proceedings of International Joint Conference on Artificial Intelligence, Montreal, Canada, pp. 1878–1885, 1995.Google Scholar
  23. 23.
    A.P. Pentland and A. Liu, “Modeling and prediction of human,” Behavior Neural Computation, 11(2), 1999.Google Scholar
  24. 24.
    N. Oliver and A.P. Pentland, “A graphical models for driver behavior recognition in a smartcar,” in Proceedings of Intelligent Vehicles, Detroit, Michigan, October 2000.Google Scholar
  25. 25.
    D. Mitrovic, “Reliable method for driving events recognition,” IEEE Transactions on Intelligent Transportation Systems, 6(2), 198–205, 2005.CrossRefGoogle Scholar
  26. 26.
    R. Simmons, B. Browning, Y. Zhang, and V. Sadekar, “Learning to predict driver route and destination intent,” in Proceedings of the 9th International IEEE Conference on Intelligent Transportation Systems, Toronto, Canada, September 17–20, 2006.Google Scholar
  27. 27.
    A. Rakotonirainy and R. Tay, “Driver inattention detection through intelligent analysis of readily available sensors,” in Proceedings of The 7th International IEEE Conference on Intelligent Transportation Systems, Washingtong DC, October 3–6, 2004.Google Scholar
  28. 28.
    A. Rakotonirainy and R. Tay, “In-vehicle ambient intelligent transport systems (i-vaits): towards an integrated research,” in Proceedings of The 7th International IEEE Conference on Intelligent Transportation Systems, Washingtong DC, October 3–6, 2004.Google Scholar
  29. 29.
    C. Brooks, A. Rakotonirainy1, and F. Maire, “Reducing driver distraction through software,” in Proceedings Australasian Road Safety Research Policing Education Conference, Wellington, New Zealand, November 2005.Google Scholar
  30. 30.
    L.M. Bergasa, J. Nuevo, M.A. Sotelo, R. Barea, and E. Lopez, “Visual monitoring of driver inattention”. Chapter 2 in this volume.Google Scholar
  31. 31.
    Y. Liang, M.L. Reyes, and J.D. Lee, “Real-time detection of driver cognitive distraction using support vector machines,” IEEE Transactions on Intelligent Transportation Systems, 8(2), 340–350, 2007.CrossRefGoogle Scholar
  32. 32.
    Applied Science Lab Inc., Eye Tracking System Instruction Manual – Model 504, Applied Science Group, Bedford, MA, 2001.Google Scholar
  33. 33.
    C.D. Wickens, S.E. Gordon, and Y. Liu, An Introduction to Human Factors Engineering, Addison–Wesley, New York, 1998.Google Scholar
  34. 34.
    M.A. Recarte and L.M. Nunes, “Mental workload while driving: effects on visual search, discrimination, and decision making,” Journal of Experimental Psychology: Applied, 9(2), 119–137, 2003.CrossRefGoogle Scholar
  35. 35.
    J. Quinlan, C4.5: Programs for Machine Learning, Morgan Kaufmann, San Mateo, CA, 1993.Google Scholar
  36. 36.
    D. Chapman and L.P. Kaelbling, “Input generalization in delayed reinforcement learning: An algorithm and performance comparisons,” in Proceedings of the Twelfth International Joint Conference on Artificial Intelligence (IJCAI-91), Sydney, Australia, pp. 726–731, August 1991.Google Scholar
  37. 37.
    Y. Freund and R.E. Shapire, “A decision-theoretic generalization of on-line learning and an application to boosting,” Journal of Computer and System Sciences, 55(1), 119–139, 1997.zbMATHCrossRefMathSciNetGoogle Scholar
  38. 38.
    Y. Freund and R.E. Shapire, “A short introduction to boosting,” Journal of Japanese Society for Artificial Intelligence, 14(5), 771–780, 1999.Google Scholar
  39. 39.
    D.C. Montgomery and G.C. Runger, Applied Statistics and Probability for Engineers, Wiley, New York, 4th edition, 2006.Google Scholar
  40. 40.
    X.S. Zheng, G.W. McConkie, and Y.-C. Tai, “The effect of secondary tasks on drivers’ scanning behavior,” in Proceedings of the 47nd Annual Meeting of the Human Factors and Ergonomics Society, pp. 1900–1903, October 2003.Google Scholar
  41. 41.
    D. Kahneman, Attention and Effort, Prentice-Hall, Englewood Cliffs, NJ, 1973.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Yilu Zhang
    • 1
  • Yuri Owechko
    • 2
  • Jing Zhang
    • 1
  1. 1.R&D CenterGeneral Motors CooperationWarrenUSA
  2. 2.HRL LaboratoriesLLC.MalibuUSA

Personalised recommendations