Skip to main content
SpringerLink
Log in

An embedded device-oriented fatigue driving detection method based on a YOLOv5s

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Currently, most fatigue driving detection methods rely on complex neural networks whose feasibility in hardware implementation needs to be further improved. This paper proposes an embedded device-oriented fatigue driving detection method based on a lightweight YOLOv5s. Firstly, a YOLOv5s face detection network with a parametric-free attention mechanism is designed to enhance the focus on face regions during face detection. Then, a practical facial landmark detector model is improved by integrating multi-scale feature fusion with Ghost module, which can adapt to the variations brought by different scale targets. Next, a fatigue determination approach is investigated by using multiple features of the face. Finally, experiments of the proposed detection model with the public YawDD dataset are implemented on the PC platform and the embedded device, respectively. The experimental results demonstrate that the proposed method achieves a detection accuracy of 95.3% and a processing speed of 22FPS on the PC platform. Meanwhile, the hardware test on an Orange Pi5 embedded device achieves a detection accuracy of 93.3% and a processing speed of 12FPS, which has good prospects for applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

The datasets generated and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Dasgupta A, Rahman D, Routray A (2018) A smartphone-based drowsiness detection and warning system for automotive drivers. IEEE Trans Intell Transp Syst 20(11):4045–4054

    Article  Google Scholar 

  2. Wang Z, Hong Q, Wang X (2020) A memristive circuit implementation of eyes state detection in fatigue driving based on biological long short-term memory rule. IEEE/ACM Trans Comput Biol Bioinf 18(6):2218–2229

    Article  ADS  Google Scholar 

  3. Shang Y, Yang M, Cui J, Cui L, Huang Z, Li X (2023) Driver emotion and fatigue state detection based on time series fusion. Electronics 12(1):26

    Article  Google Scholar 

  4. Bose R, Wang H, Dragomir A, Thakor NV, Bezerianos A, Li J (2019) Regression-based continuous driving fatigue estimation: toward practical implementation. IEEE Trans Cogn Dev Syst 12(2):323–331

    Article  Google Scholar 

  5. Wijnands JS, Thompson J, Nice KA, Aschwanden GD, Stevenson M (2020) Real-time monitoring of driver drowsiness on mobile platforms using 3d neural networks. Neural Comput Appl 32:9731–9743

    Article  Google Scholar 

  6. Abosaq HA, Ramzan M, Althobiani F, Abid A, Aamir KM, Abdushkour H, Irfan M, Gommosani ME, Ghonaim SM, Shamji V et al (2022) Unusual driver behavior detection in videos using deep learning models. Sensors 23(1):311

    Article  PubMed  PubMed Central  ADS  Google Scholar 

  7. Zhao G, He Y, Yang H, Tao Y (2022) Research on fatigue detection based on visual features. IET Image Proc 16(4):1044–1053

    Article  Google Scholar 

  8. Martinez-Alpiste I, Golcarenarenji G, Wang Q, Alcaraz-Calero JM (2021) A dynamic discarding technique to increase speed and preserve accuracy for yolov3. Neural Comput Appl 33(16):9961–9973

    Article  Google Scholar 

  9. Roy AM, Bose R, Bhaduri J (2022) A fast accurate fine-grain object detection model based on yolov4 deep neural network. Neural Comput Appl 34(5):3895–3921

  10. Li X, Hong L, Wang J-C, Liu X (2019) Fatigue driving detection model based on multi-feature fusion and semi-supervised active learning. IET Intel Transp Syst 13(9):1401–1409

    Article  Google Scholar 

  11. Ragusa E, Gianoglio C, Zunino R, Gastaldo P (2021) Random-based networks with dropout for embedded systems. Neural Comput Appl 33:6511–6526

    Article  Google Scholar 

  12. Amor RD, Colomer A, Monteagudo C, Naranjo V (2021) A deep embedded refined clustering approach for breast cancer distinction based on DNA methylation. Neural Comput Appl 34(13):10243–10255

  13. Zhang Z, Luo D, Rasim Y, Li Y, Meng G, Xu J, Wang C (2016) A vehicle active safety model: vehicle speed control based on driver vigilance detection using wearable EEG and sparse representation. Sensors 16(2):242

    Article  PubMed  PubMed Central  ADS  Google Scholar 

  14. Katsis CD, Exarchos TP, Papaloukas C, Goletsis Y, Fotiadis DI, Sarmas I (2007) A two-stage method for MUAP classification based on EMG decomposition. Comput Biol Med 37(9):1232–1240

    Article  PubMed  Google Scholar 

  15. Ma J-X, Shi L-C, Lu B-L (2014) An EOG-based vigilance estimation method applied for driver fatigue detection. Neurosci Biomed Eng (Discontin) 2(1):41–51

    Article  Google Scholar 

  16. Jia H, Xiao Z, Ji P (2022) Real-time fatigue driving detection system based on multi-module fusion. Comput Graph 108:22–33

    Article  Google Scholar 

  17. Wang M, Jeong N-T, Kim K-S, Choi S, Yang S, You S, Lee J, Suh M (2016) Drowsy behavior detection based on driving information. Int J Automot Technol 17:165–173

    Article  CAS  Google Scholar 

  18. Hailin W, Hanhui L, Zhumei S (2010) Fatigue driving detection system design based on driving behavior. In: 2010 international conference on optoelectronics and image processing, vol 110. IEEE, pp 549–552

  19. Sandberg D, Wahde M (2008) Particle swarm optimization of feedforward neural networks for the detection of drowsy driving. In: 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence). IEEE, pp 788–793

  20. Wang K, Ma Y, Huang J, Zhang C (2019) Driving performance of heavy-duty truck drivers under different fatigue levels at signalized intersections. In: CICTP 2019, pp 581–592

  21. Zandi AS, Quddus A, Prest L, Comeau FJ (2019) Non-intrusive detection of drowsy driving based on eye tracking data. Transp Res Rec 2673(6):247–257

    Article  Google Scholar 

  22. Owens JM, Dingus TA, Guo F, Fang Y, Perez M, McClafferty J, Tefft BC (2018) Estimating the prevalence and crash risk of drowsy driving using data from a large-scale naturalistic driving study. Technical report

  23. Choudhary M, Tiwari V, Uduthalapally V (2021) Iris presentation attack detection based on best-k feature selection from yolo inspired roi. Neural Comput Appl 33(11):5609–5629

    Article  Google Scholar 

  24. Zhang K, Zhang Z, Li Z, Qiao Y (2016) Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process Lett 23(10):1499–1503

    Article  ADS  Google Scholar 

  25. Li Z-Z, Zeng Q-H, Li X-D, Yu Y (2019) Face detection technology based on combining skin color model with improved adaboost algorithm. In: 2019 IEEE 4th international conference on signal and image processing (ICSIP). IEEE, pp 381–384

  26. Qin H, Yan J, Li X, Hu X (2016) Joint training of cascaded CNN for face detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3456–3465

  27. Hai L, Guo H (2020) Face detection with improved face r-cnn training method. In: Proceedings of the 3rd international conference on control and computer vision, pp 22–25 (2020)

  28. Deng Z, Yang R, Lan R, Liu Z, Luo X (2020) Se-iyolov3: an accurate small scale face detector for outdoor security. Mathematics 8(1):93

    Article  Google Scholar 

  29. Yang W, Jiachun Z (2018) Real-time face detection based on yolo. In: 2018 1st IEEE international conference on knowledge innovation and invention (ICKII). IEEE, pp 221–224

  30. Liu Y, Liu R, Wang S, Yan D, Peng B, Zhang T (2022) Video face detection based on improved ssd model and target tracking algorithm. J Web Eng 21(2):545–567

  31. Jiang C, Ma H, Li L (2022) Irnet: an improved retinanet model for face detection. In: 2022 7th international conference on image, vision and computing (ICIVC). IEEE, pp 129–134

  32. Wang J, Wang N, Li L, Ren Z (2020) Real-time behavior detection and judgment of egg breeders based on yolo v3. Neural Comput Appl 32:5471–5481

    Article  Google Scholar 

  33. Gai R, Chen N, Yuan H (2021) A detection algorithm for cherry fruits based on the improved yolo-v4 model. Neural Comput Appl 35(19):13895–13906

  34. Yang L, Zhang R-Y, Li L, Xie X (2021) Simam: a simple, parameter-free attention module for convolutional neural networks. In: International conference on machine learning. PMLR, pp 11863–11874

  35. Guo X, Li S, Yu J, Zhang J, Ma J, Ma L, Liu W, Ling H (2019) Pfld: a practical facial landmark detector. arXiv preprint arXiv:1902.10859

  36. Han K, Wang Y, Tian Q, Guo J, Xu C, Xu C (2020) Ghostnet: more features from cheap operations. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1580–1589

  37. Yang S, Song X, Zhang L, Yu J (2017) The anti-fatigue driving system design based on the eye blink detect. In: Seventh international conference on electronics and information engineering, vol 10322. SPIE, pp. 406–410

  38. Soukupova T, Cech J (2016) Eye blink detection using facial landmarks. In: 21st computer vision winter workshop, Rimske Toplice, Slovenia, p 2

  39. Abtahi S, Omidyeganeh M, Shirmohammadi S, Hariri B (2014) Yawdd: a yawning detection dataset. In: Proceedings of the 5th ACM multimedia systems conference, pp 24–28

  40. Horrey WJ, Wickens CD (2007) In-vehicle glance duration: distributions, tails, and model of crash risk. Transp Res Rec 2018(1):22–28

    Article  Google Scholar 

  41. Yang S, Luo P, Loy C-C, Tang X (2016) Wider face: a face detection benchmark. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5525–5533

  42. Kumar A, Kaur A, Kumar M (2019) Face detection techniques: a review. Artif Intell Rev 52:927–948

    Article  Google Scholar 

  43. Wu W, Qian C, Yang S, Wang Q, Cai Y, Zhou Q (2018) Look at boundary: a boundary-aware face alignment algorithm. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2129–2138

Download references

Acknowledgements

Financially supported by the National Natural Science Foundation of China (Grant No. 61673190) and the Fundamental Research Funds for the Central Universities under Grant CCNU22JC011.

Author information

Authors and Affiliations

  1. School of Computer Science, China University of Geosciences, Wuhan, 430074, People’s Republic of China

    Jiaxiang Qu

  2. College of Physical Science and Technology, Central China Normal University, Wuhan, 430079, People’s Republic of China

    Ziming Wei & Yimin Han

Authors
  1. Jiaxiang Qu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ziming Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yimin Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ziming Wei.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qu, J., Wei, Z. & Han, Y. An embedded device-oriented fatigue driving detection method based on a YOLOv5s. Neural Comput & Applic 36, 3711–3723 (2024). https://doi.org/10.1007/s00521-023-09255-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-09255-9

Keywords

Search

Navigation