Skip to main content
SpringerLink
Log in

Driver Fatigue Monitoring Based on Facial Multifeature Fusion

  • Conference paper
  • First Online:
Computer Applications (CCF NCCA 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1960))

Included in the following conference series:

  • 91 Accesses

Abstract

In response to the problems of low efficiency and inaccuracy of existing fatigue driving recognition and detection algorithms, this paper proposes a machine learning-based driver fatigue detection algorithm. First, an adaptive median filtering algorithm is used to denoise the collected images, and then the Summed-area table is used to calculate its Har Like eigenvalue. Second, the Adaboost face detection and recognition algorithm and the tracking algorithm that predicts the relocation of the target position are used to quickly extract the features of the tracked face, and the improved cascade regression Tree model is used to locate the face feature points. The support vector machine algorithm is used to classify the collected eye feature values, setting a threshold to determine whether the eyes are in a closed state, and fusing the feature values of the mouth and head. Finally, through a fatigue detection algorithm based on multifeature weighted summation, repeated experimental simulations were conducted on the trained dataset to determine the weight values of the eye parameters as 0.55, the mouth parameters as 0.25, and the head parameters as 0.20. Based on the differences in the weight values of the feature values obtained from the simulation experiments, the fatigue level was divided into awake, mild fatigue, moderate fatigue, and severe fatigue. Repeated experimental data testing shows that the method has greatly improved the accuracy and recognition speed of sleep-deprived driving recognition and detection. Tests on a dataset of simulated driving have shown that our model has an average accuracy of approximately 95.1%.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Zhu, M., Chen, J., Li, H., Liang, F., Han, L., Zhang, Z.: Vehicle driver drowsiness detection method using wearable EEG based on convolution neural network. Neural Comput. Appl. 33(20), 13965–13980 (2021)

    Article  Google Scholar 

  2. Ling, Y., Luo, R., Dong, X., Weng, X.: Driver eye location and state estimation based on a robust model and data augmentation. IEEE Access 9, 67219–67231 (2021)

    Article  Google Scholar 

  3. Arefnezhad, S., Samiee, S., Eichberger, A., Frühwirth, M., Kaufmann, C., Klotz, E.: Applying deep neural networks for multi-level classification of driver drowsiness using vehicle-based measures. Expert Syst. Appl. 162, 113778 (2020)

    Article  Google Scholar 

  4. Li, K., Gong, Y., Ren, Z.: A fatigue driving detection algorithm based on facial multi-feature fusion. IEEE Access 8, 101244–101259 (2020)

    Article  Google Scholar 

  5. Pilataxi, J., Vinan, W., Chavez, D.: Design and implementation of a driving assistance system in a car-like robot when fatigue in the user is detected. IEEE Latin Am. Trans. 14(2), 457–462 (2016)

    Article  Google Scholar 

  6. Sun, W., Zhang, X., Wang, J., He, J., Peeta, S.: Blink number forecasting based on improved Bayesian fusion algorithm for fatigue driving detection. Math. Probl. Eng. (2015)

    Google Scholar 

  7. Ji, Y., Wang, S., Lu, Y., Wei, J., Zhao, Y.: Eye and mouth state detection algorithm based on contour feature extraction. J. Electron. Imaging 27(5), 1 (2018)

    Article  Google Scholar 

  8. Baohu, Y., Li, L.A.N., Yong, D.: Face detection based on skin color segmentation and AdaBoost algorithm. Sci. Technol. Eng. 13(11), 3119–3125 (2013)

    Google Scholar 

  9. Jo, S.P., Park, J.H., Lee, I., Kim, S.H., Kim, M.U., Kim, J.T.: Driver Fatigue Evaluation Methods for Commercial Vehicles (2013)

    Google Scholar 

  10. Pan, T., Wang, H., Si, H., Li, Y., Shang, L.: Identification of pilots’ fatigue status based on electrocardiogram signals. Sensors 21(9), 3003 (2021)

    Article  Google Scholar 

  11. Kim, J., Kim, S., Han, Y., et al.: Night vehicle detection using variable haar-like feature. J. Measur. Sci. Instrum. 2(04), 337–340 (2011)

    Google Scholar 

  12. Segundo, M.P.P., Silva, L., Bellon, O.R.P., Queirolo, C.C.: Automatic face segmentation and facial landmark detection in range images. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 40(5), 1319–1330 (2010)

    Google Scholar 

  13. Vishnu, R., Vishvaragul, S., Srihari, P., Nithiavathy, R.: Driver drowsiness detection system with opencv and keras. J. Phys. Conf. Ser. (2021)

    Google Scholar 

  14. Zhang, J.Y., Qiu, W.W., Fu, H.J., Zhang, M.T., Ma, Q.G.: Review of techniques for driver fatigue detection. Appl. Mech. Mater. 433, 928–931 (2013)

    Article  Google Scholar 

  15. Reddy, B., Kim, Y.H., Yun, S., Seo, C., Jang, J.: Real-time driver drowsiness detection for embedded system using model compression of deep neural networks. In: Proceedings of the Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA, 21–26 July 2017, pp. 438–445 (2017)

    Google Scholar 

  16. Lyu, J., Yuan, Z., Chen, D.: Long-term multigranularity deep framework for driver drowsiness detection. arXiv 2018, arXiv:1801.02325 (2018)

    Google Scholar 

  17. Chen, C., Seo, H., Jun, C.H., Zhao, Y.: Pavement crack detection and classification based on fusion feature of LBP and PCA with SVM. Int. J. Pavement Eng. 23(9), 3274–3283 (2022)

    Article  Google Scholar 

  18. Wei, W., Weidong, M., Mengdan, F., Han, Q.: Real-timeface recognition based on selective detection and multiscale matching. Comput. Eng. Des. (2018)

    Google Scholar 

  19. Dua, M., Shakshi, Singla, R., Raj, S., Jangra, A.: Deep CNN models-based ensemble approach to driver drowsiness detection. Neural Comput. Appl. 33, 3155–3168 (2021)

    Google Scholar 

  20. Doudou, M., Bouabdallah, A., Berge-Cherfaoui, V.: Driver drowsiness measurement technologies: current research, market solutions, and challenges. Int. J. Intell. Transp. Syst. Res. 18, 297–319 (2020)

    Google Scholar 

  21. Shen, H.M., Xu, M.H.: Design and implementation of embedded driver fatigue monitor system. Science and Engineering Research Center. Proceedings of 2015 International Conference on Artificial Intelligence and Industrial Engineering (AIIE 2015) (2015)

    Google Scholar 

  22. Zhou, C., Li, J.: A real-time driver fatigue monitoring system based on lightweight convolutional neural network (2021)

    Google Scholar 

  23. Chen, S., Sun, Y., Zhang, H., Liu, Q., Lv, X., Mei, X.: Speech fatigue detection based on deep learning. J. Phys. Conf. Ser. (2022)

    Google Scholar 

  24. Mollicone, D., et al.: Predicting performance and safety based on driver fatigue. Accid. Anal. Prev. 126, 142–145 (2019)

    Article  Google Scholar 

  25. Ning, Z., et al.: When deep reinforcement learning meets 5G-enabled vehicular networks: a distributed offloading framework for traffic big data. IEEE Trans. Ind. Informat. 16(2), 1352–1361 (2019)

    Article  Google Scholar 

  26. Amin, H.U., Yusoff, M.Z., Ahmad, R.F.: A novel approach based on wavelet analysis and arithmetic coding for automated detection and diagnosis of epileptic seizure in EEG signals using machine learning techniques. Biomed. Signal Process. Control 56, 101707 (2020)

    Article  Google Scholar 

  27. Tsipouras, M.G.: Spectral information of EEG signals with respect to epilepsy classification. EURASIP J. Adv. Signal Process. 2019(1), 1–17 (2019). https://doi.org/10.1186/s13634-019-0606-8

    Article  Google Scholar 

  28. Tuncer, T., Dogan, S., Páawiak, P., Acharya, U.R.: Automated arrhythmia detection using novel hexadecimal local pattern and multilevel wavelet transform with ECG signals. Knowl.-Based Syst. 186, 104923 (2019)

    Article  Google Scholar 

  29. Celona, L., Mammana, L., Bianco, S., Schettini, R.: A multi-task CNN framework for driver face monitoring. In: Proceedings of the 2018 IEEE 8th International Conference on Consumer Electronics-Berlin (ICCE-Berlin), Berlin, Germany, 2–5 September 2018, pp. 1–4 (2018)

    Google Scholar 

  30. Zuojin, L., Shengbo, L., Renjie, L., Bo, C., Jinliang, S.: Online detection of driver fatigue using steering wheel anglesfor real driving conditions. Sensors 17(3), 495 (2017)

    Article  Google Scholar 

  31. Jiang, Y., Guo, S., Deng, S.: Denoising and chaotic feature extraction of electrocardial signals for driver fatigue detection by Kolmogorov entropy. J. Dyn. Syst. Meas. Contr. 141(2), 245–258 (2018)

    Google Scholar 

  32. Knapik, M., Cyganek, B.: Driver’s fatigue recognition based on yawn detection in thermal images. Neurocomputing 338(2019), 274–292 (2019)

    Article  Google Scholar 

  33. Qin, X., Yiwei, S., Xuefeng, Z.: Progress of driving fatigue monitoring methods based on PERCLOS. Autom. Technol. Appl. 27(6), 43–46 (2008)

    Google Scholar 

  34. Buendia, R., Forcolin, F., Karlsson, J., Arne Sjöqvist, B., Anund, A., Candefjord, S.: Deriving heart rate variability indices from cardiac monitoring—an indicator of driver sleepiness. J. Crash Prev. Injury Control 20(3), 249–254 (2019)

    Google Scholar 

  35. Sun, Y., Tang, Z.: Traffic accident analysis and control countermeasures of fatigue driving on expressway. J. Phys. Conf. Ser. (2021)

    Google Scholar 

  36. Chen, L., Zhi, X., Wang, H., Wang, G., Zhou, Z., Yazdani, A., Zheng, X.: Driver fatigue detection via differential evolution extreme learning machine technique. Electronics 9(11), 1850 (2020)

    Article  Google Scholar 

  37. Tang, X., Guo, F., Shen, J., Du, T.: Facial landmark detection by semi-supervised deep learning. Neurocomputing 297, 22–32 (2018)

    Article  Google Scholar 

  38. Patacchiola, M., Cangelosi, A.: Head pose estimation in the wild using convolutional neural networks and adaptive gradient methods. Pattern Recognit. 71, 132–143 (2017)

    Article  Google Scholar 

  39. Zhao, Z., Zhou, N., Zhang, L., Yan, H., Xu, Y., Zhang, Z.: Driver fatigue detection based on convolutional neural networks using EM-CNN. Comput. Intell. Neurosci. (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Xi’an Siyuan University, Shaanxi, 710038, China

    Jie Wang, Weiwei Zhang & Jinlong Zhao

  2. Northwestern University, Shaanxi, 71027, China

    Jun Guo

Authors
  1. Jie Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weiwei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jinlong Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weiwei Zhang .

Editor information

Editors and Affiliations

  1. Suzhou University, Suzhou, China

    Min Zhang

  2. Tsinghua University, Beijing, China

    Bin Xu

  3. Suzhou University of Science and Technology, Suzhou, China

    Fuyuan Hu

  4. Institute of Information Engineering, CAS, Beijing, China

    Junyu Lin

  5. Harbin University of Science and Technology, Harbin, China

    Xianhua Song

  6. National Academy of Guo Ding Institute of Data Science, Beijing, China

    Zeguang Lu

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, J., Zhang, W., Zhao, J., Guo, J. (2024). Driver Fatigue Monitoring Based on Facial Multifeature Fusion. In: Zhang, M., Xu, B., Hu, F., Lin, J., Song, X., Lu, Z. (eds) Computer Applications. CCF NCCA 2023. Communications in Computer and Information Science, vol 1960. Springer, Singapore. https://doi.org/10.1007/978-981-99-8761-0_9

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8761-0_9

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8760-3

  • Online ISBN: 978-981-99-8761-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation