Abstract
In the paper we address a problem of estimating a physical state of an observed person by means of analysing facial portrait captured in thermal spectrum. The algorithm consists of facial regions detection combined with tracking and individual features classification. We focus on eyes and mouth state estimation. The detectors are based on Haar-like features and AdaBoost, previously applied to visible-band images. Returned face region is subject to eyes and mouth detection. Further, extracted regions are filtered using Gabor filter bank and the resultant features are classified. Finally, classifiers’ responses are integrated and the decision about driver’s physical state is taken. By using thermal image we are able to capture eyes and mouth states in very adverse lighting conditions, in contrast to the visible-light approaches. Experiments performed on manually annotated video sequences have shown that the proposed approach is accurate and can be a part of current Advanced Driver Assistant Systems.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alioua, N., Amine, A., Rziza, M.: Driver’s fatigue detection based on yawning extraction. Int. J. Veh. Technol. 2014, 7 (2014). Article ID: 678786
Andrysiak, T., Choras, M.: Image retrieval based on hierarchical Gabor filters. Int. J. Appl. Math. Comput. Sci. 15(4), 471–480 (2005)
Azim, T., Jaffar, M.A., Mirza, A.M.: Fully automated real time fatigue detection of drivers through fuzzy expert systems. Appl. Soft Comput. 18, 25–38 (2014)
Berglund, B., Rossi, G.B., Townsend, J.T., Pendrill, L.R.: Measurement with Persons: Theory, Methods, and Implementation Areas, p. 422. Psychology Press, New York (2013)
Bianconi, F., Fernandez, A.: Evaluation of the effects of Gabor filter parameters on texture classification. Pattern Recogn. 40(12), 3325–3335 (2007)
Burduk, R.: The AdaBoost algorithm with the imprecision determine the weights of the observations. In: Nguyen, N.T., Attachoo, B., Trawiński, B., Somboonviwat, K. (eds.) ACIIDS 2014. LNCS (LNAI), vol. 8398, pp. 110–116. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-05458-2_12
Craye, C., Rashwan, A., Kamel, M.S., Karray, F.: A multi-modal driver fatigue and distraction assessment system. Int. J. Intell. Transp. Syst. Res. 14(3), 173–194 (2016). https://doi.org/10.1007/s13177-015-0112-9
Cyganek, B., Gruszczynski, S.: Hybrid computer vision system for drivers’ eye recognition and fatigue monitoring. Neurocomputing 126, 78–94 (2014)
Daza, I.G., Bergasa, L.M., Bronte, S., Yebes, J.J., Almazán, J., Arroyo, R.: Fusion of optimized indicators from Advanced Driver Assistance Systems (ADAS) for driver drowsiness detection. Sensors 14, 1106–1131 (2014)
Forczmański, P.: Performance evaluation of selected thermal imaging-based human face detectors. In: Kurzynski, M., Wozniak, M., Burduk, R. (eds.) CORES 2017. AISC, vol. 578, pp. 170–181. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-59162-9_18
Forczmański, P., Kutelski, K.: Driver drowsiness estimation by means of face depth map analysis. In: Pejaś, J., El Fray, I., Hyla, T., Kacprzyk, J. (eds.) ACS 2018. AISC, vol. 889, pp. 396–407. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-03314-9_34
Fu, R., Wang, H., Zhao, W.: Dynamic driver fatigue detection using hidden Markov model in real driving condition. Exp. Syst. Appl. 63, 397–411 (2016)
Hussien, M.N., Lye, M., Fauzi, M.F.A., Seong, T.C., Mansor, S.: Comparative analysis of eyes detection on face thermal images. In: 2017 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), Kuching, pp. 385–389 (2017)
Jo, J., Lee, S.J., Park, K.R., Kim, I.J., Kim, J.: Detecting driver drowsiness using feature-level fusion and user-specific classification. Exp. Syst. Appl. 41(4), 1139–1152 (2014)
Josephin, J., Lakshmi, C., James, S.: A review on the measures and techniques adapted for the detection of driver drowsiness. In: IOP Conference Series: Materials Science and Engineering, vol. 993, p. 012101 (2020). https://doi.org/10.1088/1757-899X/993/1/012101
Knipling, R.R., Wierwille, W.W.: Vehicle-based drowsy driver detection: current status and future prospects. In: Moving Toward Deployment. Proceedings of the IVHS America Annual Meeting. 2 Volumes, vol. 1, pp. 245–256 (1994)
Knapik, M., Cyganek, B.: Driver’s fatigue recognition based on yawn detection in thermal images. Neurocomputing 338, 274–292 (2019)
Kong, W., Zhou, L., Wang, Y., Zhang, J., Liu, J., Gao, S.: A system of driving fatigue detection based on machine vision and its application on smart device. J. Sens. 2015, 11 (2015). https://doi.org/10.1155/2015/548602. Article ID: 548602
Krishnasree, V., Balaji, N., Rao, P.S.: A real time improved driver fatigue monitoring system. WSEAS Trans. Sig. Process. 10, 146–155 (2014)
Małecki, K., Forczmański, P., Nowosielski, A., Smoliński, A., Ozga, D.: A new benchmark collection for driver fatigue research based on thermal, depth map and visible light imagery. In: Burduk, R., Kurzynski, M., Wozniak, M. (eds.) CORES 2019. AISC, vol. 977, pp. 295–304. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-19738-4_30
Małecki, K., Nowosielski, A., Forczmański, P.: Multispectral data acquisition in the assessment of driver’s fatigue. In: Mikulski, J. (ed.) TST 2017. CCIS, vol. 715, pp. 320–332. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66251-0_26
Mitas, A., Czapla, Z., Bugdol, M., Rygula, A.: Registration and evaluation of biometric parameters of the driver to improve road safety, pp. 71–79. Scientific Papers of Transport, Silesian University of Technology (2010)
Nissimagoudar, P.C., Nandi, A.V., Gireesha, H.M.: A feature extraction and selection method for EEG based driver alert/drowsy state detection. In: Abraham, A., Jabbar, M.A., Tiwari, S., Jesus, I.M.S. (eds.) SoCPaR 2019. AISC, vol. 1182, pp. 297–306. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-49345-5_31
Nowosielski, A.: Vision-based solutions for driver assistance. J. Theor. Appl. Comput. Sci. 8(4), 35–44 (2014)
Nowosielski, A., Forczmański, P.: Touchless typing with head movements captured in thermal spectrum. Pattern Anal. Appl. 22(3), 841–855 (2018). https://doi.org/10.1007/s10044-018-0741-0
Poster, D., Hu, S., Nasrabadi, N.M., Riggan, B.S.: An examination of deep-learning based landmark detection methods on thermal face imagery. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 980–987 (2019)
Sacco, M., Farrugia, R.A.: Driver fatigue monitoring system using support vector machines. In: 2012 5th International Symposium on Communications, Control and Signal Processing, Rome, pp. 1–5 (2012)
Satish, K., Lalitesh, A., Bhargavi, K., Prem, M.S., Anjali, T.: Driver drowsiness detection. In: 2020 International Conference on Communication and Signal Processing (ICCSP), pp. 0380–0384 (2020)
Smolensky, M.H., Di Milia, L., Ohayon, M.M., Philip, P.: Sleep disorders, medical conditions, and road accident risk. Accid. Anal. Prev. 43(2), 533–548 (2011)
Trutschel, U., Sirois, B., Sommer, D., Golz, M., Edwards, D.: PERCLOS: an alertness measure of the past. In: Proceedings of the Sixth International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design, pp. 172–179 (2017)
Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004). https://doi.org/10.1023/B:VISI.0000013087.49260.fb
Virginia Tech Transportation Institute: Day or Night, Driving while Tired a Leading Cause of Accidents (2013). http://www.vtnews.vt.edu/articles/2013/04/041513-vtti-fatigue.html
Weller, G., Schlag, B.: Road user behavior model. Deliverable D8 project RIPCORD-ISERET, 6 Framework Programme of the European Union (2007). http://ripcord.bast.de
Zhang, Y., Hua, C.: Driver fatigue recognition based on facial expression analysis using local binary patterns. Optik Int. J. Light Electron Opt. 126(23), 4501–4505 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Forczmański, P., Smoliński, A. (2021). Supporting Driver Physical State Estimation by Means of Thermal Image Processing. In: Paszynski, M., Kranzlmüller, D., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M. (eds) Computational Science – ICCS 2021. ICCS 2021. Lecture Notes in Computer Science(), vol 12746. Springer, Cham. https://doi.org/10.1007/978-3-030-77977-1_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-77977-1_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-77976-4
Online ISBN: 978-3-030-77977-1
eBook Packages: Computer ScienceComputer Science (R0)