Skip to main content

Advertisement

SpringerLink
Log in

Evaluation of driver drowsiness using respiration analysis by thermal imaging on a driving simulator

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper, a new non-intrusive driver drowsiness detection method is introduced based on respiration analysis using facial thermal imaging. Drowsiness is the cause of many driving accidents all over the world. Drivers’ respiration system undergoes significant changes from wakefulness to drowsiness and can be used to detect drowsiness. Current respiration measurement methods are intrusive and uncomfortable making respiration the least measured vital sign during driving. In this paper, a new method is presented based on facial thermal imaging to analyze drivers’ respiration signal non-intrusively. Thirty subjects are tested in a car simulator. They are fully awake at the beginning and experience drowsiness during the tests. The mean and the standard deviation of the respiration rate and the inspiration-to-expiration time ratio are extracted from the subjects’ respiration signal. To detect drowsiness, the Support Vector Machine (SVM) and the K-Nearest Neighbor (KNN) classifiers are used. The Observer Rating of Drowsiness method is used for scoring the drowsiness level and validating the proposed method. The performance and the results of both methods are presented and compared. The results indicate that drowsiness can be detected with the accuracy of 90%, sensitivity of 92%, specificity of 85%, and precision of 91%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Åkerstedt T, Gillberg M (1990) Subjective and objective sleepiness in the active individual. Int J Neurosci 52:29–37. https://doi.org/10.3109/00207459008994241

    Article  Google Scholar 

  2. Alkali AH, Saatchi R, Elphick H, Burke D (2013) Facial tracking in thermal images for real-time noncontact respiration rate monitoring. In: 2013 European Modelling symposium. Pp 265–270

  3. Arefnezhad S, Samiee S, Eichberger A, Nahvi A (2019) Driver drowsiness detection based on steering wheel data applying adaptive Neuro-fuzzy feature selection. Sensors 19:943–957. https://doi.org/10.3390/s19040943

    Article  Google Scholar 

  4. Balandong RP, Ahmad RF, Saeed MA (2018) A review on EEG-based automatic sleepiness detection systems for driver. IEEE Access 6:22908–22919. https://doi.org/10.1109/ACCESS.2018.2811723

    Article  Google Scholar 

  5. Bartula M, Tigges T, Muehlsteff J (2013) Camera-based system for contactless monitoring of respiration. In: Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp 2672–2675

    Google Scholar 

  6. Bernacchia N, Scalise L, Casacanditella L et al (2014) Non contact measurement of heart and respiration rates based on Kinect™. In: IEEE International Symposium on Medical Measurements and Applications, pp 1–5

    Google Scholar 

  7. Chauvin R, Hamel M, Briere S et al (2016) Contact-free respiration rate monitoring using a pan-tilt thermal camera for stationary bike Telerehabilitation sessions. IEEE Syst J 10:1046–1055. https://doi.org/10.1109/JSYST.2014.2336372

    Article  Google Scholar 

  8. Chekmenev SY, Farag AA, Miller WM, Essock EA (2009) Multiresolution approach for noncontact measurements of arterial pulse using thermal imaging. In: Augmented vision perception in infrared. Springer, Berlin, pp 87–112

    Chapter  Google Scholar 

  9. Chowdhury A, Shankaran R, Kavakli M, Haque M (2018) Sensor applications and physiological features in drivers ’ drowsiness detection : a review. IEEE Sensors J 18:3055–3067. https://doi.org/10.1109/JSEN.2018.2807245

    Article  Google Scholar 

  10. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20:273–297. https://doi.org/10.1023/A:1022627411411

    Article  MATH  Google Scholar 

  11. Danisman T, Bilasco IM, Djeraba C, Ihaddadene N (2010) Drowsy driver detection system using eye blink patterns. In: International Conference on Machine and Web Intelligence, pp 230–233

    Google Scholar 

  12. Dasgupta A, George A, Happy SL, Routray A, Shanker T (2013) An on-board vision based system for drowsiness detection in automotive drivers. Int J Adv Eng Sci Appl Math 5:94–103. https://doi.org/10.1007/s12572-013-0086-2

    Article  Google Scholar 

  13. Daubechies I, Lu J, Wu HT (2011) Synchrosqueezed wavelet transforms: an empirical mode decomposition-like tool. Appl Comput Harmon Anal 30:243–261. https://doi.org/10.1016/j.acha.2010.08.002

    Article  MathSciNet  MATH  Google Scholar 

  14. De Naurois C, Bourdin C, Bougard C, Vercher J (2018) Adapting artificial neural networks to a specific driver enhances detection and prediction of drowsiness. Accid Anal Prev 121:118–128. https://doi.org/10.1016/j.aap.2018.08.017

    Article  Google Scholar 

  15. Dhupati LS, Kar S, Rajaguru A, Routray A (2010) A novel drowsiness detection scheme based on speech analysis with validation using simultaneous EEG recordings. In: 2010 IEEE international conference on automation science and engineering. Pp 917–921

  16. Douglas NJ, White DP, Pickett CK, Weil JV, Zwillich CW (1982) Respiration during sleep in normal man. Thorax 37:840–844

    Article  Google Scholar 

  17. Fei J, Pavlidis I (2010) Thermistor at a distance: unobtrusive measurement of breathing. IEEE Trans Biomed Eng 57:988–998. https://doi.org/10.1109/TBME.2009.2032415

    Article  Google Scholar 

  18. Fei J, Pavlidis I, Murthy J (2009) Thermal vision for sleep apnea monitoring. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp 1084–1091

    Google Scholar 

  19. Flores MJ, Armingol JM, de la Escalera A (2011) Driver drowsiness detection system under infrared illumination for an intelligent vehicle. IET Intell Transp Syst 5:241–251. https://doi.org/10.1049/iet-its.2009.0090

    Article  Google Scholar 

  20. Fors C, Ahlstrom C, Anund A, Fors C (2018) A comparison of driver sleepiness in the simulator and on the real road the real road. J Transp Saf Secur 10:72–87. https://doi.org/10.1080/19439962.2016.1228092

    Article  Google Scholar 

  21. Friedrichs F, Yang B (2010) Camera-based drowsiness reference for driver state classification under real driving conditions. In: IEEE Intelligent Vehicles Symposium, pp 101–106

    Google Scholar 

  22. Gade R, Moeslund TB (2014) Thermal cameras and applications: a survey. Mach Vis Appl 25:245–262. https://doi.org/10.1007/s00138-013-0570-5

    Article  Google Scholar 

  23. Garbey M, Sun N, Merla A, Pavlidis I (2007) Contact-free measurement of cardiac pulse based on the analysis of thermal imagery. IEEE Trans Biomed Eng 54:1418–1426

    Article  Google Scholar 

  24. Gault TR, Blumenthal N, Farag AA, Starr T (2010) Extraction of the Superficial Facial Vasculature , Vital Signs Waveforms and Rates Using Thermal Imaging. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, pp 1–8

    Google Scholar 

  25. González-Ortega D, Díaz-Pernas FJ, Martínez-Zarzuela M, Antón-Rodríguez M (2019) A physiological sensor-based android application synchronized with a driving simulator for driver monitoring. Sensors 19:399. https://doi.org/10.3390/s19020399

    Article  Google Scholar 

  26. Guede-fernández F, Fernández-chimeno M, Ramos-Castro J, Garcia-Gonzalez MA (2019) Driver drowsiness detection based on respiratory signal analysis. IEEE Access 7:81826–81838. https://doi.org/10.1109/ACCESS.2019.2924481

    Article  Google Scholar 

  27. Harris C, Stephens M (1988) A combined corner and edge detector. Alvey vision conference, In, pp 147–151

    Google Scholar 

  28. Hoddes E, Zarcone V, Smythe H, Phillips R, Dement WC (1973) Quantification of sleepiness: a new approach. Psychophysiology 10:431–436

    Article  Google Scholar 

  29. Igasaki T, Nagasawa K, Murayama N, Hu Z (2015) Drowsiness estimation under driving environment by heart rate variability and / or breathing rate variability with logistic regression analysis. In: 8th international conference on BioMedical engineering and informatics. Pp 189–193

  30. Igasaki T, Nagasawa K, Akbar IA, Kubo N (2016) Sleepiness classification by thoracic respiration using support vector machine. In: The 2016 biomedical engineering international conference, pp 1–5

    Google Scholar 

  31. Kartsch VJ, Benatti S, Schiavone PD et al (2018) A sensor fusion approach for drowsiness detection in wearable ultra-low-power systems. Inf Fusion 43:66–76. https://doi.org/10.1016/j.inffus.2017.11.005

    Article  Google Scholar 

  32. Kiashari SEH, Nahvi A, Homayounfard A, Bakhoda H (2018) Monitoring the variation in driver respiration rate from wakefulness to drowsiness : a non-intrusive method for drowsiness detection using thermal imaging. J Sleep Sci 3:1–9 Retrieved from http://jss.tums.ac.ir/index.php/jss/article/view/110

    Google Scholar 

  33. Kotsiantis SB (2007) Supervised machine learning: a review of classification techniques. Informatica 31:249–268. https://doi.org/10.1115/1.1559160

    Article  MathSciNet  MATH  Google Scholar 

  34. Langroodi AK, Nahvi A (2018) Design and implementation reinforcement learning algorithm for driver drowsiness detection. SAE Int J Commer Veh 11:57–64. https://doi.org/10.4271/02-11-01-0005

    Article  Google Scholar 

  35. Lee B, Lee B, Chung W (2014) Mobile healthcare for automatic driving sleep-onset detection using wavelet-based EEG and respiration signals. Sensors 14:17915–17936. https://doi.org/10.3390/s141017915

    Article  Google Scholar 

  36. Mahmoodi M, Nahvi A (2019) Driver drowsiness detection based on classification of surface electromyography features in a driving simulator. Proc Inst Mech Eng Part H J Eng Med. https://doi.org/10.1177/0954411919831313

  37. Mcdonald AD, Lee JD, Schwarz C, Brown TL (2018) A contextual and temporal algorithm for driver drowsiness detection. Accid Anal Prev 113:25–37. https://doi.org/10.1016/j.aap.2018.01.005

    Article  Google Scholar 

  38. Meng C, Shi-wu L, Wen-cai S et al (2019) Drowsiness monitoring based on steering wheel status. Transp Res part D Transp Environ 66:95–103. https://doi.org/10.1016/j.trd.2018.07.007

    Article  Google Scholar 

  39. Murthy R, Pavlidis I, Tsiamyrtzis P (2004) Touchless monitoring of breathing function. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, pp 1196–1199

    Chapter  Google Scholar 

  40. Murthy JN, Van Jaarsveld J, Fei J et al (2009) Thermal infrared imaging: a novel method to monitor airflow during polysomnography. Sleep 32:1521–1527

    Article  Google Scholar 

  41. Patel M, Lal SKL, Kavanagh D, Rossiter P (2011) Applying neural network analysis on heart rate variability data to assess driver fatigue. Expert Syst Appl 38:7235–7242. https://doi.org/10.1016/j.eswa.2010.12.028

    Article  Google Scholar 

  42. Pereira CB, Yu X, Czaplik M, Rossaint R, Blazek V, Leonhardt S (2015) Remote monitoring of breathing dynamics using infrared thermography. Biomed Opt Express 6:4378–4394. https://doi.org/10.1364/BOE.6.004378

    Article  Google Scholar 

  43. Philip RC, Ram S, Gao X, Rodríguez JJ (2014) A comparison of tracking algorithm performance for objects in wide area imagery. In: Southwest Symposium on Image Analysis and Interpretation, pp 109–112

    Chapter  Google Scholar 

  44. Rahman A, Sirshar M, Khan A (2015) Real time drowsiness detection using eye blink monitoring. In: National Software Engineering Conference, pp 1–7

    Google Scholar 

  45. Rezaei M, Klette R (2014) Look at the driver, look at the road: no distraction! No accident! Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit:129–136. https://doi.org/10.1109/CVPR.2014.24

  46. Rodríguez-Ibáñez N, García-González MA, Fernández-Chimeno M (2011) Drowsiness detection by thoracic effort signal analysis in real driving environments. In: 33rd annual international conference of the IEEE EMBS. Pp 6055–6058

  47. Rodríguez-Ibáñez N, García-González MA, Fernández-Chimeno M et al (2013) Synchrosqueezing index for detecting drowsiness based on the respiratory effort signal. In: XIII Mediterranean Conference on Medical and Biological Engineering and Computing, pp 965–968

    Google Scholar 

  48. Rohit F, Kulathumani V, Kavi R et al (2017) Real-time drowsiness detection using wearable, lightweight brain sensing headbands. IET Intell Transp Syst 11:255–263. https://doi.org/10.1049/iet-its.2016.0183

    Article  Google Scholar 

  49. Sahayadhas A, Sundaraj K, Murugappan M (2012) Detecting driver drowsiness based on sensors: a review. Sensors 12:16937–16953. https://doi.org/10.3390/s121216937

    Article  Google Scholar 

  50. Samiee S, Azadi S, Kazemi R, Nahvi A, Eichberger A (2014) Data fusion to develop a driver drowsiness detection system with robustness to signal loss. Sensors 14:17832–17847. https://doi.org/10.3390/s140917832

    Article  Google Scholar 

  51. Schleicher R, Galley N, Briest S, Galley L (2008) Blinks and saccades as indicators of fatigue in sleepiness warnings : looking tired ? Ergonomics 51:982–1010. https://doi.org/10.1080/00140130701817062

    Article  Google Scholar 

  52. Shuyan H, Gangtie Z (2009) Driver drowsiness detection with eyelid related parameters by support vector machine. Expert Syst Appl 36:7651–7658. https://doi.org/10.1016/j.eswa.2008.09.030

    Article  Google Scholar 

  53. Singh S (2015) Critical reasons for crashes investigated in the National Motor Vehicle Crash Causation Survey

  54. Tarassenko L, Villarroel M, Guazzi A, Jorge J, Clifton DA, Pugh C (2014) Non-contact video-based vital sign monitoring using ambient light and auto-regressive models. Physiol Meas 35:807–831. https://doi.org/10.1088/0967-3334/35/5/807

    Article  Google Scholar 

  55. Tashakori M, Nahvi A, Shahidian A, et al (2018) Estimation of Driver Drowsiness Using Blood Perfusion Analysis of Facial Thermal Images in a Driving Simulator. J Sleep Sci 3:45–52. Retrieved from http://jss.tums.ac.ir/index.php/jss/article/view/122

  56. Tateno S, Guan X, Cao R, Qu Z (2018) Developement of Drowsiness Detection System Based on Respiration Changes Using Heart Rate Monitoring. In: 57th Annual Conference of the Society of Instrument and Control Engineers of Japan, pp 1664–1669

    Google Scholar 

  57. Trinder J, Whitworth F, Kay A, Wilkin P (1992) Respiratory instability during sleep onset. J Appl Physiol 73:2462–2469

    Article  Google Scholar 

  58. Wang X, Xu C (2016) Driver drowsiness detection based on non-intrusive metrics considering individual specifics. Accid Anal Prev 95:350–357. https://doi.org/10.1016/j.aap.2015.09.002

    Article  Google Scholar 

  59. Wang MS, Jeong NT, Kim KS et al (2016) Drowsy behavior detection based on driving information. Int J Automot Technol 17:165–173. https://doi.org/10.1007/s12239

    Article  Google Scholar 

  60. Warwick B, Symons N, Chen X, Xiong K (2015) Detecting driver drowsiness using wireless Wearables. In: IEEE 12th international conference on Mobile ad hoc and sensor systems, pp 585–588

    Chapter  Google Scholar 

  61. Wiegand DM, McClafferty J, McDonald SE, Hanowski RJ (2009) Development and evaluation of a naturalistic observer rating of drowsiness protocol. Virginia Tech. Virginia Tech Transportation Institute

  62. Wilburta LQ, Pooler M, Tamparo CD, Dahl BM (2010) Vital signs and measurements. In: Delmar’s comprehensive medical assisting: administrative and clinical competencies, 4th Editio, pp 564–598

    Google Scholar 

  63. Wu H, Rubinstein M, Shih E et al (2012) Eulerian video magnification for revealing subtle changes in the world. ACM Trans Graph 31:1–8

    Article  Google Scholar 

  64. Xie A (2012) Effect of sleep on breathing - why recurrent apneas are only seen during sleep. J Thorac Dis 4:194–197. https://doi.org/10.3978/j.issn.2072-1439.2011.04.04

    Article  Google Scholar 

  65. Zhang K, Zhang L, Yang M-H, Zhang D (2014) Fast tracking via dense Spatio-temporal context learning. In: European conference on computer vision. Springer, Cham, pp 127–141

    Google Scholar 

  66. Zhen Z, Tsiamyrtzis P, Pavlidis I (2008) The segmentation of the supraorbital vessels in thermal imagery. In: IEEE 5th international conference on advanced video and signal based surveillance. AVSS 2008:237–244

    Google Scholar 

Download references

Acknowledgements

This paper is based upon a work supported by the Cognitive Science and Technology Council (CSTC) under Grant No. 1307.

Author information

Authors and Affiliations

  1. Faculty of Mechanical Engineering, K.N. Toosi University of Technology, Tehran, Iran

    Serajeddin Ebrahimian Hadi Kiashari, Ali Nahvi, Hamidreza Bakhoda, Amirhossein Homayounfard & Masoumeh Tashakori

Authors
  1. Serajeddin Ebrahimian Hadi Kiashari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ali Nahvi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hamidreza Bakhoda
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Amirhossein Homayounfard
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Masoumeh Tashakori
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Serajeddin Ebrahimian Hadi Kiashari.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kiashari, S.E.H., Nahvi, A., Bakhoda, H. et al. Evaluation of driver drowsiness using respiration analysis by thermal imaging on a driving simulator. Multimed Tools Appl 79, 17793–17815 (2020). https://doi.org/10.1007/s11042-020-08696-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08696-x

Keywords

Search

Navigation