Skip to main content
SpringerLink
Log in

A smart analysis of driver fatigue and drowsiness detection using convolutional neural networks

  • 1202: Multimedia Tools for Digital Twin​
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Automotive industry experiences multiple injuries in our everyday life. The increasing road accident rate is due to driver drowsiness, such as fatigue and insomnia. This research is intended primarily to diagnose exhaustion and drowsiness, utilizing the deep learning models like convolution neural network. A research project has been initiated in the Kingdom of Saudi Arabia to address the problem of identifying the driver’s drowsiness state. In this article, we present a real-time driver disturbance monitoring method using Convolutional Neural Network (CNN). Detection is done using state-of-the-art CNN models such as InceptionV3, VGG16, and ResNet50. We have analyzed that InceptionV3 has 90.70% with 0.6931 loss, the VGG16 has 39.87% with 0.6931 loss, and the ResNet50 has 93.69% accuracy with 0.6931 loss. So, the ResNet50 model has the best accuracy as compared to all other deep learning models. Using sleepy and active human face image collection, we present the output review of the CNN models. We expanded device deployment and performance review of the best-performing CNN model among the evaluated models, utilizing our custom dataset to enhance real-time efficiency. This unique data collection is similar to the driver end scenarios because it includes not just the side view, but also the driver’s front view, greatly enhancing the performance. Extending this work into a commodity and its large-scale production will also render a significant contribution to the Kingdom’s economy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Bayar B, Stamm MC (2016) A deep learning approach to universal image manipulation detection using a new convolutional layer. In: Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security, pp 5–10

    Chapter  Google Scholar 

  2. Chang W-J, Chen L-B, Chiou Y-Z (2018) Design and implementation of a drowsiness-fatigue-detection system based on wearable smart glasses to increase road safety. IEEE Trans Consum Electron 64(4):461–469

    Article  Google Scholar 

  3. Fu S, Lu L, Li H, Li Z, Wu W, Paul A, Jeon G, Yang X (2020) A real-time super-resolution method based on convolutional neural networks. Circ Syst Signal Process 39(2):805–817

    Article  Google Scholar 

  4. Giusti A, Cireşan DC, Masci J, Gambardella LM, Schmidhuber J (2013) Fast image scanning with deep max-pooling convolutional neural networks. In: 2013 IEEE International Conference on Image Processing. IEEE, pp 4034–4038

    Chapter  Google Scholar 

  5. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167

  6. Jabbar R, Al-Khalifa K, Kharbeche M, Alhajyaseen W, Jafari M, Jiang S (2018) Real-time driver drowsiness detection for android application using deep neural networks techniques. Proc Comput Sci 130:400–407

    Article  Google Scholar 

  7. Jabbar R, Shinoy M, Kharbeche M, Al-Khalifa K, Krichen M, Barkaoui K (2020) Driver drowsiness detection model using convolutional neural networks techniques for android application," arXiv preprint arXiv:2002.03728

  8. Kapoor K, Pamula R, Murthy SV (2020) Real-time driver distraction detection system using convolutional neural networks. In: Proceedings of ICETIT 2019. Springer, pp 280–291

    Chapter  Google Scholar 

  9. Kundinger T, Yalavarthi PK, Riener A, Wintersberger P, Schartmüller C (2020) Feasibility of smart wearables for driver drowsiness detection and its potential among different age groups. Int J Pervasive Comput Commun

  10. Lee KW, Yoon HS, Song JM, Park KR (2018) Convolutional neural network-based classification of driver’s emotion during aggressive and smooth driving using multi-modal camera sensors. Sensors 18(4):957

    Article  Google Scholar 

  11. Lee KH, Kim W, Choi HK, Jang BT (2019) a study on feature extraction methods used to estimate a driver’s level of drowsiness. In: 2019 21st International Conference on Advanced Communication Technology (ICACT). IEEE, pp 710–713

    Chapter  Google Scholar 

  12. Leshno M, Lin VY, Pinkus A, Schocken S (1993) Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural Netw 6(6):861–867

    Article  Google Scholar 

  13. Li Z, Chen L, Peng J, Wu Y (2017) Automatic detection of driver fatigue using driving operation information for transportation safety. Sensors 17(6):1212

    Article  Google Scholar 

  14. Li Z, Yang Q, Chen S, Zhou W, Chen L, Song L (2019) A fuzzy recurrent neural network for driver fatigue detection based on steering-wheel angle sensor data. Int J Distrib Sens Netw 15(9):1550147719872452

    Google Scholar 

  15. Li J, Li H, Umer W, Wang H, Xing X, Zhao S, Hou J (2020) Identification and classification of construction equipment operators' mental fatigue using wearable eye-tracking technology. Autom Constr 109:103000

    Article  Google Scholar 

  16. Liu W, Qian J, Yao Z, Jiao X, Pan J (2019) Convolutional two-stream network using multi-facial feature fusion for driver fatigue detection. Future Internet 11(5):115

    Article  Google Scholar 

  17. Mann JD, Jaworski EG (1970) Minimizing loss of indoleacetic acid during purification of plant extracts. Planta 92(4):285–291

    Article  Google Scholar 

  18. Mehta S, Dadhich S, Gumber S, Jadhav Bhatt A (2019) Real-time driver drowsiness detection system using eye aspect ratio and eye closure ratio. Available at SSRN 3356401

  19. Minhas AA, Jabbar S, Farhan M, ul Islam MN (2019) Smart methodology for safe life on roads with active drivers based on real-time risk and behavioral monitoring. J Ambient Intell Humaniz Comput:1–13

  20. Monteiro AN, August A, Hanafusa H (1996) Evidence for a transcriptional activation function of BRCA1 C-terminal region. Proc Natl Acad Sci 93(24):13595–13599

    Article  Google Scholar 

  21. Ngxande M, Tapamo J-R, Burke M (2019) Detecting inter-sectional accuracy differences in driver drowsiness detection algorithms. arXiv preprint arXiv:1904.12631

  22. Nissimagoudar PC, Nandi AV (2020) Precision enhancement of driver assistant system using EEG based driver consciousness analysis & classification. In: Computational network application tools for performance management. Springer, pp 247–257

    Chapter  Google Scholar 

  23. Nosseir A, Hamad A, Wahdan A (2020) Detecting drivers’ fatigue in different conditions using real-time non-intrusive system. In: Fourth international congress on information and communication technology. Springer, pp 141–151

    Chapter  Google Scholar 

  24. Park S, Pan F, Kang S, Yoo CD (2016) Driver drowsiness detection system based on feature representation learning using various deep networks. In: Asian Conference on Computer Vision. Springer, pp 154–164

    Google Scholar 

  25. Riaz F, Jabbar S, Sajid M, Ahmad M, Naseer K, Ali N (2018) A collision avoidance scheme for autonomous vehicles inspired by human social norms. Comput Electr Eng 69:690–704

    Article  Google Scholar 

  26. Riaz F, Khadim S, Rauf R, Ahmad M, Jabbar S, Chaudhry J (2018) A validated fuzzy logic inspired driver distraction evaluation system for road safety using artificial human driver emotion. Comput Netw 143:62–73

    Article  Google Scholar 

  27. Ross PJ, Ross PJ (1988) Taguchi techniques for quality engineering: loss function, orthogonal experiments, parameter and tolerance design (no. TS156 R12). McGraw-Hill, New York

    Google Scholar 

  28. Schmidtmann G, Kennedy GJ, Orbach HS, Loffler G (2012) Non-linear global pooling in the discrimination of circular and non-circular shapes. Vis Res 62:44–56

    Article  Google Scholar 

  29. Shahverdy M, Fathy M, Berangi R, Sabokrou M (2020) Driver behavior detection and classification using deep convolutional neural networks. Expert Syst Appl:113240

  30. Tayab Khan M, Anwar H, Ullah F, Ur Rehman A, Ullah R, Iqbal A, Lee BH, Kwak KS (2019) Smart real-time video surveillance platform for drowsiness detection based on eyelid closure. Wirel Commun Mob Comput 2019:1–9

    Article  Google Scholar 

  31. Vijayan V, Sherly E (2019) Real time detection system of driver drowsiness based on representation learning using deep neural networks. J Intell Fuzzy Syst 36(3):1977–1985

    Article  Google Scholar 

  32. Willmott CJ, Matsuura K (2005) Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Clim Res 30(1):79–82

    Article  Google Scholar 

  33. Zhang Y-D, Muhammad K, Tang C (2018) Twelve-layer deep convolutional neural network with stochastic pooling for tea category classification on GPU platform. Multimed Tools Appl 77(17):22821–22839

    Article  Google Scholar 

  34. Zhou F et al (2020) Driver fatigue transition prediction in highly automated driving using physiological features. Expert Syst Appl:113204

Download references

Acknowledgement

This research, including all funds and support for equipment, was fully supported by Al Yamamah University, Riyadh, Kingdom of Saudi Arabia.

Author information

Authors and Affiliations

  1. Department of Computer Engineering, College of Engineering and Architecture, Al Yamamah University, Riyadh, Kingdom of Saudi Arabia

    Abid Ali Minhas

  2. Department of Computer Science, The University of Faisalabad, Faisalabad, Pakistan

    Sohail Jabbar

  3. Department of Computer Science, COMSATS University Islamabad, Sahiwal Campus, Sahiwal, Pakistan

    Muhammad Farhan

  4. Department of Electrical Engineering, Bahria University, Islamabad, Pakistan

    Muhammad Najam ul Islam

Authors
  1. Abid Ali Minhas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sohail Jabbar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muhammad Farhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Muhammad Najam ul Islam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muhammad Farhan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Minhas, A.A., Jabbar, S., Farhan, M. et al. A smart analysis of driver fatigue and drowsiness detection using convolutional neural networks. Multimed Tools Appl 81, 26969–26986 (2022). https://doi.org/10.1007/s11042-022-13193-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13193-4

Keywords

Search

Navigation