Skip to main content
SpringerLink
Log in

Driver fatigue detection based on improved YOLOv7

  • Research
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Fatigue driving is one of the main reasons threatening road traffic safety. Aiming at the problems of complex detection process, low accuracy, and susceptibility to light interference in the current driver fatigue detection algorithm, this paper proposes a driver Eye State detection algorithm based on YOLO, abbreviated as ES-YOLO. The algorithm optimizes the structure of YOLOv7, integrates the multi-scale features using the convolutional block attention mechanism (CBAM), and improves the attention to important spatial locations in the image. Furthermore, using the Focal-EIOU Loss instead of CIOU Loss to increase the attention on difficult samples and reduce the influence of sample class imbalance. Then, based on ES-YOLO, a driver fatigue detection method is proposed, and the driver fatigue judgment logic is designed to monitor the fatigue state in real-time and alarm in time to improve the accuracy of detection. The experiments on the public dataset CEW and the self-made dataset show that the proposed ES-YOLO obtained 99.0% and 98.8% mAP values, respectively, which are better than the compared algorithms. And this method achieves real-time and accurate detection of driver fatigue status. Source code is released in https://www.github/driver-fatigue-detection.git.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

No datasets were generated or analysed during the current study.

References

  1. Sikander, G., Anwar, S.: Driver fatigue detection systems. A review. IEEE Trans. Intell. Transp. Syst. 20, 2339–2352 (2019)

    Article  Google Scholar 

  2. KołOdziej, M., Tarnowski, P., Sawicki, D.J., Majkowski, A., Rak, R.J., Bala, A., Pluta, A.: Fatigue detection caused by office work with the use of EOG signal. IEEE Sens. J. 20, 15213–15223 (2020)

    Article  Google Scholar 

  3. Jap, B.T., Lal, S., Fischer, P., Bekiaris, E.: Using EEG spectral components to assess algorithms for detecting fatigue. Expert Syst. Appl. 36, 2352–2359 (2009)

    Article  Google Scholar 

  4. Zhao, L., Li, M., He, Z., Ye, S., Qin, H., Zhu, X., Dai, Z.: Data-driven learning fatigue detection system: a multimodal fusion approach of ECG (electrocardiogram) and video signals. Measurement 201, 111648 (2022)

    Article  Google Scholar 

  5. Mashayekhi, M., Moghaddam, M.: EMG-driven fatigue-based self-adapting admittance control of a hand rehabilitation robot. J. Biomech. 138, 111104 (2022)

    Article  Google Scholar 

  6. Lin, B., Wu, P., Chen, C.: 2D/3D-display auto-adjustment switch system. IEEE J. Biomed. Health Inform. 22, 799–805 (2018)

    Article  Google Scholar 

  7. Dogan, S., Tuncer, I., Baygin, M., Tuncer, T.: A new hand-modeled learning framework for driving fatigue detection using EEG signals. Neural Comput. Appl. 35, 14837–14854 (2023)

    Article  Google Scholar 

  8. Zhang, J., Wu, Y., Chen, Y., Wang, J., Huang, J., Zhang, Q.: Ubi-fatigue: toward ubiquitous fatigue detection via contactless sensing. IEEE Internet Things J. 9, 14103–14115 (2022)

    Article  Google Scholar 

  9. Chen, J., Wang, H., Hua, C.: Electroencephalography based fatigue detection using a novel feature fusion and extreme learning machine. Cognitive Syst. Res. 52, 715–728 (2018)

    Article  Google Scholar 

  10. Li, Z., Li, S., Li, R., Cheng, B., Shi, J.: Driver fatigue detection using approximate entropic of steering wheel angle from real driving data. Int. J. Robot. Autom. 17, 495 (2017)

    Google Scholar 

  11. Forsman, P., Vila, B., Short, R., Mott, C., Dongen, H.: Efficient driver drowsiness detection at moderate levels of drowsiness. Accid. Anal. Prev. 50, 341–350 (2013)

    Article  Google Scholar 

  12. Yi, Y., Zhou, Z., Zhang, W., Zhou, M., Yuan, Y., Li, C.: Fatigue detection algorithm based on eye multifeature fusion. IEEE Sens. J. 23, 7949–7955 (2023)

    Article  Google Scholar 

  13. Jia, H., Xiao, Z., Ji, P.: Real-time fatigue driving detection system based on multi-module fusion. Comput. Graph. 108, 22–33 (2022)

    Article  Google Scholar 

  14. Du, G., Zhang, L., Su, K., Wang, X., Teng, S., Liu, P.: A multimodal fusion fatigue driving detection method based on heart rate and PERCLOS. IEEE trans. Intell. Transp. Syst. 23, 21810–21820 (2022)

    Article  Google Scholar 

  15. Sun, Z., Miao, Y., Jeon, J., Kong, Y., Park, G.: Facial feature fusion convolutional neural network for driver fatigue detection. Eng. App. Artif. Intell. 126, 106981 (2023)

    Article  Google Scholar 

  16. Knapik, M., Cyganek, B.: Driver’s fatigue recognition based on yawn detection in thermal images. Neurocomputing 338, 274–292 (2019)

    Article  Google Scholar 

  17. Yang, H., Liu, L., Min, W., Yang, X., Xiong, X.: Driver yawning detection based on subtle facial action recognition. IEEE Trans. Multimedia 23, 572–583 (2021)

    Article  Google Scholar 

  18. Li, C., Li, L., Jiang, H., et al: YOLOv6: A single-stage object detection framework for industrial applications. arXiv (2022) https://doi.org/10.48550/arXiv.2209.02976

  19. Wang, C., Bochkovskiy, A., Liao, H.: YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In: IEEE computer society conference on computer vision and pattern recognition, 2023. CVPR2023. IEEE, pp 7464–7475 (2023)

  20. Woo, S., Park, J., Lee, J., Kweon, I.: CBAM: convolutional block attention module. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer vision—ECCV 2018. ECCV 2018. Lecture notes in computer science, vol. 11211. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_1

    Chapter  Google Scholar 

  21. Tolstikhin, I., Houlsby, N., Kolesnikov, A., et al: MLP-mixer: an all-MLP ARCHITECTURE FOR VISION. arXiv (2021) https://doi.org/10.48550/arXiv.2105.01601

  22. Zheng, Z., Wang, P., Ren, D., Liu, W., Ye, R., Hu, Q., Zuo, W.: Enhancing geometric factors in model learning and inference for object detection and instance segmentation. IEEE Trans. Cybern. 52, 8574–8586 (2022)

    Article  Google Scholar 

  23. Zhang, Y., Ren, W., Zhang, Z., Jia, Z., Wang, L., Tan, T.: Focal and efficient IOU loss for accurate bounding box regression. Neurocomputing 506, 146–157 (2022)

    Article  Google Scholar 

  24. Mu, Z., Jin, L., Yin, J., Wang, Q.: Research on a driver fatigue detection model based on image processing. Comput. Intell. Neurosci. (2022). https://doi.org/10.22967/HCIS.2022.12.017

    Article  Google Scholar 

  25. Zhou, M., Zhang, H., Zhang, H., Yi, Y.: An improved random forest algorithm-based fatigue recognition with multiphysical feature. IEEE Sens. J. 23, 26195–26201 (2023)

    Article  Google Scholar 

  26. Ghoddoosian, R., Galib, M., Athitsos, V.: A realistic dataset and baseline temporal model for early drowsiness detection. In: IEEE Computer society conference on computer vision and pattern recognition workshops, 2019. CVPRW2019. IEEE, pp 178–187 (2019)

  27. Liu, Z., Jiang, C., Li, S., Wu, M., Cao, W., Hao, M.: Eye state detection based on weight binarization convolution neural network and transfer learning. Appl. Soft Comput. 109, 107565 (2021)

    Article  Google Scholar 

  28. Zhang, H., Cisse, M., Dauphin, Y., Lopez-Paz D.: mixup: beyond empirical risk minimization. arXiv (2017) https://doi.org/10.48550/arXiv.1710.09412

Download references

Acknowledgements

This work was supported by the Tianjin “Project+Team” Key Training Special Project under Grant XB 202007 and Tianjin Transportation Technology Development Project Plan under Grant 2021-36.

Funding

This study was supported by The Tianjin “Project + Team” Key Training Special Project (XB 202007); Tianjin Transportation Technology Development Project Plan (2021-36).

Author information

Authors and Affiliations

  1. Tiangong University, Tianjin, 300387, China

    Xianguo Li, Xueyan Li & Zhenqian Shen

  2. Tianjin Key Laboratory of Optoelectronic Detection Technology and System, Tianjin, 300387, China

    Xianguo Li

  3. Tianjin Railway Traffic Operation Group Co, Tianjin, 300392, China

    Guangmin Qian

Authors
  1. Xianguo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xueyan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhenqian Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guangmin Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. The data analysis was done by Li, X.G. The systems and experiments were designed by Li, X.Y. The paper was revised by Shen, Z.Q., and the research was conducted by Qian, G.M.

Corresponding author

Correspondence to Zhenqian Shen.

Ethics declarations

Conflict of interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, X., Li, X., Shen, Z. et al. Driver fatigue detection based on improved YOLOv7. J Real-Time Image Proc 21, 75 (2024). https://doi.org/10.1007/s11554-024-01455-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11554-024-01455-3

Keywords

Search

Navigation