A Lightweight Framework for Multi-device Integration and Multi-sensor Fusion to Explore Driver Distraction

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11483)


Driver distraction is a major challenge in road traffic and major cause of accidents. Vehicle industry dedicates increasing amounts of resources to better quantify the various activities of drivers resulting in distraction. Literature has shown that significant causes for driver distraction are tasks performed by drivers which are not related to driving, like using multimedia interfaces or glancing at co-drivers. One key aspect of the successful implementation of distraction prevention mechanisms is to know when the driver performs such auxiliary tasks. Therefore, capturing these tasks with appropriate measurement equipment is crucial. Especially novel quantification approaches combining data from different sensors and devices are necessary for comprehensively determining causes of driver distraction. However, as a literature review has revealed, there is currently a lack of lightweight frameworks for multi-device integration and multi-sensor fusion to enable cost-effective and minimally obtrusive driver monitoring with respect to scalability and extendibility. This paper presents such a lightweight framework which has been implemented in a demonstrator and applied in a small real-world study involving ten drivers performing simple distraction tasks. Preliminary results of our analysis have indicated a high accuracy of distraction detection for individual distraction tasks and thus the framework’s usefulness. The gained knowledge can be used to develop improved mechanisms for detecting driver distraction through better quantification of distracting tasks.


Driver distraction Driver attention Lightweight framework Multi-device integration Multi-sensor fusion 



Parts of this study were funded by the Austrian Research Promotion Agency (FFG) under project number 866781 (FFG FEMTech Project GENDrive). The authors would further like to acknowledge the financial support of the COMET K2 – Competence Centers for Excellent Technologies Programme of the Federal Ministry for Transport, Innovation and Technology (bmvit), the Federal Ministry for Digital, Business and Enterprise (bmdw), the Austrian Research Promotion Agency (FFG), the Province of Styria and the Styrian Business Promotion Agency (SFG).


  1. 1.
    NHTSA Risky Driving. Accessed 20 Nov 2018
  2. 2.
    Study on good practices for reducing road safety risks caused by road user distractions. Accessed 20 Nov
  3. 3.
    Research Note on Distracted Driving. Accessed 20 Nov
  4. 4.
    Thomas, P., Morris, A., Talbot, R., Fagerlind, H.: Identifying the causes of road crashes in Europe. Ann. Adv. Automot. Med. 57(13), 13–22 (2013)Google Scholar
  5. 5.
    Payre, W., Cestac, J., Dang, N.-T., Vienne, F., Delhomme, P.: Impact of training and in-vehicle task performance on manual control recovery in an automated car. Transp. Res. Part F: Traffic Psychol. Behav. 46(A), 216–227 (2017)CrossRefGoogle Scholar
  6. 6.
    Zeeb, K., Buchner, A., Schrauf, M.: What determines the take-over time? An integrated model approach of driver take-over after automated driving. Accid. Anal. Prev. 78, 212–221 (2015)CrossRefGoogle Scholar
  7. 7.
    Youtube Driver Distraction. Accessed 20 Nov 2018
  8. 8.
    Youtube Understanding Driver Distraction. Accessed 20 Nov 2018
  9. 9.
  10. 10.
  11. 11.
    Regan, M.A., Hallett, C., Gordon, C.P.: Driver distraction and driver inattention: definition, relationship and taxonomy. Accid. Anal. Prev. 43(5), 1771–1781 (2011)CrossRefGoogle Scholar
  12. 12.
    Sussman, E.D., Bishop, H., Madnick, B., Walters, R.: Driver inattention and highway safety. Transp. Res. Rec. 1047, 40–48 (1985)Google Scholar
  13. 13.
    Young, K., Regan, M.: Driver distraction: a review of the literature. In: Faulks, I.J., Regan, M., Stevenson, M., Brown, J., Porter, A., Irwin, J.D. (eds.) Distracted driving, pp. 379–405. Australasian College of Road Safety, Sydney (2007)Google Scholar
  14. 14.
    Dong, Y., Hu, Z., Uchimura, K., Murayama, N.: Driver inattention monitoring system for intelligent vehicles: a review. IEEE Trans. Intell. Transp. Syst. 12(2), 596–614 (2011)CrossRefGoogle Scholar
  15. 15.
    Horberry, T., Anderson, J., Regan, M.A., Triggs, T.J., Brown, J.: Driver distraction: the effects of concurrent in-vehicle tasks, road environment complexity and age on driving performance. Accid. Anal. Prev. 38(1), 185–191 (2006)CrossRefGoogle Scholar
  16. 16.
    Klauer, S., Dingus, T., Neale, V., Sudweeks, J., Ramsey, D.: The Impact of Driver Inattention on Near-Crash/Crash Risk: An Analysis Using the 100-Car Naturalistic Driving Study Data. Virginia Tech Transportation Institute, Blacksburg (2006)Google Scholar
  17. 17.
    D’Orazio, T., Leo, M., Guaragnella, C., Distante, A.: A visual approach for driver inattention detection. Pattern Recogn. 40(8), 2341–2355 (2007)CrossRefGoogle Scholar
  18. 18.
    Fletcher, L., Zelinsky, A.: Driver inattention detection based on eye gaze—road event correlation. Int. J. Robot. Res. 28(6), 774–801 (2009)CrossRefGoogle Scholar
  19. 19.
    Swan, M.: Emerging patient-driven health care models: an examination of health social networks, consumer personalized medicine and quantified self-tracking. Int. J. Environ. Res. Public Health 6(2), 492–525 (2009)CrossRefGoogle Scholar
  20. 20.
    Swan, M.: The quantified self: fundamental disruption in big data science and biological discovery. Big data 1(2), 85–99 (2013)CrossRefGoogle Scholar
  21. 21.
    Stocker, A., Kaiser, C., Fellmann, M.: Quantified vehicles. Bus. Inf. Syst. Eng. 59(2), 125–130 (2017)CrossRefGoogle Scholar
  22. 22.
  23. 23.
    SAE J1939 Standards Collection on the Web: Content. Accessed 27 Nov 2018
  24. 24.
    Pandian, P.S., Mohanavelu, K., Safeer, K.P., Kotresh, T.M., Shakunthala, D.T., Gopal, P., Padaki, V.C.: Smart Vest: wearable multi-parameter remote physiological monitoring system. Med. Eng. Phys. 30(4), 466–477 (2008)CrossRefGoogle Scholar
  25. 25.
    Gellersen, H.W., Schmidt, A., Beigl, M.: Multi-sensor context-awareness in mobile devices and smart artifacts. Mob. Netw. Appl. 7(5), 341–351 (2002)CrossRefGoogle Scholar
  26. 26.
    Ramos, F.B.A., Lorayne, A., Costa, A.A.M., de Sousa, R.R., et al.: Combining smartphone and smartwatch sensor data in activity recognition approaches: an experimental evaluation. In: Proceedings of the 28th International Conference on Software Engineering and Knowledge Engineering, SEKE 2016, Redwood City, pp. 267–272 (2016)Google Scholar
  27. 27.
    Shoaib, M., Bosch, S., Scholten, H., Havinga, P.J., Incel, O.D.: Towards detection of bad habits by fusing smartphone and smartwatch sensors. In: Proceedings of the 2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops), St. Louis, pp. 591–596 (2015)Google Scholar
  28. 28.
    Vilarinho, T., Farshchian, B., Bajer, D.G., Dahl, O.H., et al.: A combined smartphone and smartwatch fall detection system. In: Proceedings of the 2015 IEEE International Conference on Computer and Information Technology, Liverpool, pp. 1443–1448 (2015)Google Scholar
  29. 29.
    Casilari, E., Santoyo-Ramón, J.A., Cano-García, J.M.: Analysis of a smartphone-based architecture with multiple mobility sensors for fall detection. PLoS ONE 11(12), e0168069 (2016)CrossRefGoogle Scholar
  30. 30.
    Giang, W.C., Shanti, I., Chen, H.Y.W., Zhou, A., Donmez, B.: Smartwatches vs. smartphones: a preliminary report of driver behavior and perceived risk while responding to notifications. In: Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Nottingham, pp. 154–161 (2015)Google Scholar
  31. 31.
    Liu, L., Karatas, C., Li, H., Tan, S., et al.: Toward detection of unsafe driving with wearables. In: Proceedings of the 2015 workshop on Wearable Systems and Applications, Florence, pp. 27–32 (2015)Google Scholar
  32. 32.
    De Arriba-Pérez, F., Caeiro-Rodríguez, M., Santos-Gago, J.M.: Collection and processing of data from wrist wearable devices in heterogeneous and multiple-user scenarios. Sensors 16(9), 1538 (2016)CrossRefGoogle Scholar
  33. 33.
    Kluyver, T., Ragan-Kelley, B., Pérez, F., Granger, B., et al.: Jupyter notebooks – a publishing format for reproducible computational workflows. In: Proceedings of the 20th International Conference on Electronic Publishing, pp. 87–90. IOS Press, Goettingen (2016)Google Scholar
  34. 34.
    McKinney, W.: Data structures for statistical computing in Python. In: Proceedings of the 9th Python in Science Conference, Austin, pp. 51–56 (2010)Google Scholar
  35. 35.
    Oliphant, T.E.: Guide to NumPy, 2nd edn. CreateSpace Independent Publishing Platform, USA (2015)Google Scholar
  36. 36.
    Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series FeatuRe extraction on basis of scalable hypothesis tests. Neurocomputing 307, 72–77 (2018)CrossRefGoogle Scholar
  37. 37.
    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)MathSciNetzbMATHGoogle Scholar
  38. 38.
    Kaiser, C., Stocker, A., Festl, A., Lechner, G., Fellmann, M.: A research agenda for vehicle information systems. ECIS (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Virtual Vehicle Research CenterGrazAustria
  2. 2.University of RostockRostockGermany

Personalised recommendations