Multi-Sensor Soft-Computing System for Driver Drowsiness Detection

  • Li Li
  • Klaudius Werber
  • Carlos F. Calvillo
  • Khac Dong Dinh
  • Ander Guarde
  • Andreas König
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 223)


Advanced sensing systems, sophisticated algorithms and increasing computational resources continuously enhance active safety technology for vehicles. Driver status monitoring belongs to the key components of advanced driver assistance system which is capable of improving car and road safety without compromising driving experience. This paper presents a novel approach to driver status monitoring aimed at drowsiness detection based on depth camera, pulse rate sensor and steering angle sensor. Due to NIR active illumination depth camera can provide reliable head movement information in 3D alongside eye gaze estimation and blink detection in a non-intrusive manner. Multi-sensor data fusion on feature level and multilayer neural network facilitate the classification of driver drowsiness level based on which a warning can be issued to prevent traffic accidents. The presented approach is implemented on an integrated soft-computing system for driving simulation (DeCaDrive) with multi-sensing interfaces. The classification accuracy of \(98.9\,\%\) for up to three drowsiness levels has been achieved based on data sets of five test subjects with 588-min driving sequence.


Kinect Sensor Depth Camera Scale Conjugate Gradient Multilayer Feedforward Neural Network National Highway Traffic Safety Administration 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The authors would like to thank Abhaya C. Kammara for giving support to construct the DeCaDrive system. The help from students in ISE are gratefully appreciated.


  1. 1.
    von Seelen, W.: Elektronisches Auge OPEL, Multimodaler Sensor zur Fahrzeugführung: Teilprojekt: Architektur, Rundumsicht und Objekterkennung; Abschlubericht zum 30. Juni 1997. Univ., Inst. für Neuroinformatik (1997)Google Scholar
  2. 2.
    Skribanowitz, J., Knobloch, T., Schreiter, J., König, A.: VLSI Implementation of an application-specific vision chip for overtake monitoring, real time eye tracking, and visual inspection. In: MicroNeuro, vol. 99, pp. 4552 (1999)Google Scholar
  3. 3.
    Ford Motor Company: last visited: 02.11.2012
  4. 4.
    Daimler, A.G.: last visited: 02.11.2012
  5. 5.
    Volkswagen, A.G.: last visited: 02.11.2012
  6. 6.
    Toyota Motor Corporation: last visited: 02.11.2012
  7. 7.
    Volvo Car Corporation: last visited: 02.11.2012
  8. 8.
    Li, L., Xu, Y., König, A.: Robust depth camera based multi-user eye tracking for autostereoscopic displays. In: 9th International Multi-Conference on SSD, pp. 1–6 (2012)Google Scholar
  9. 9.
    Werber, K.: Untersuchung von Fahrerassistenzsystemen zur Fahrer- Zustands- und Absichtserkennung mit Multisensorik. In: Diplomarbeit, ISE, TU Kaiserslautern (2012)Google Scholar
  10. 10.
    Li, L., Xu, Y., König, A.: Robust depth camera based eye localization for human-machine interactions. In: KES’2011, vol. 6881, p. 424 (2011)Google Scholar
  11. 11.
    Li, H., Roivainen, P., Forchheimer, R.: 3-D motion estimation in model-based facial image coding. IEEE Trans. PAMI 15(6), 545–555 (1993)Google Scholar
  12. 12.
    Face Tracking: last visited: 28.10.2012
  13. 13.
    Guarde, A.: Kinect based eye tracking for driver drowsiness detection. In: Studienarbeit, Institute of Integrated Sensor Systems (ISE), TU Kaiserslautern (2012)Google Scholar
  14. 14.
    Matsumoto, Y., Zelinsky, A.: An algorithm for real-time stereo vision implementation of head pose and gaze direction measurement. In: 4th IEEE International Conference on Automatic Face and Gesture Recognition, pp. 499–504 (2000)Google Scholar
  15. 15.
    Sherman, P.: The potential of steering wheel information to detect driver drowsiness and associated lane departure. Technical Report, Iowa State University (1996)Google Scholar
  16. 16.
    Yu, X.: Real-time nonintrusive detection of driver drowsiness: Final Report. Technical Report, University of Minnesota (2009)Google Scholar
  17. 17.
    Møller, M. F.: A scaled conjugate gradient algorithm for fast supervised learning. Neural Netw. 6(4), 525–533 (1993)Google Scholar
  18. 18.
    Moré, J.J.: The Levenberg-Marquardt algorithm: implementation and theory. Numerical Anal. Dundee (1977)Google Scholar
  19. 19.
    Feng R., Zhang, G., Cheng, B.: An on-board system for detecting driver drowsiness based on multi-sensor data fusion using Dempster-Shafer theory, In: ICNSC, pp. 897–902 (2009)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Li Li
    • 1
  • Klaudius Werber
    • 1
  • Carlos F. Calvillo
    • 1
  • Khac Dong Dinh
    • 1
  • Ander Guarde
    • 2
  • Andreas König
    • 1
  1. 1.TU Kaiserslautern, Department of Electrical and Computer EngineeringInstitute of Integrated Sensor SystemsKaiserslauternGermany
  2. 2.Faculty of Engineering of BilbaoUniversity of the Basque CountryBilbaoSpain

Personalised recommendations