A Multimodal Vigilance Monitoring System Based on Fuzzy Logic Architecture

  • Ahmed Snoun
  • Ines Teyeb
  • Olfa Jemai
  • Mourad Zaied
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10637)


This paper deals with the problem of vigilance level monitoring. A novel method of hypovigilance detection is presented in this work. It is based on the analysis of eyes’ blinking and head posture. The fusion task of both systems is achieved by the fuzzy logic technique which allows us to obtain five vigilance levels. This paper contains two key contributions. The first is the amelioration of our previous works in the classification field employing fast wavelet network classifier (FWT) by using another classification system based on a deep learning architecture. It provides more accurate results than the wavelet network classifier. The second resides in the conception of a driver alertness control system able to detect five vigilance levels which is different from previous works of the literature characterized by two, three or four levels. Experiments, using different datasets, prove the good performance of our new approach.


Head pose Fuzzy Sleep Eyes’ blinking Deep learning 



The authors would like to acknowledge the financial support of this work by grants from General Direction of Scientific Research (DGRST), Tunisia, under the ARUB program.


  1. 1.
    Picot, A., Charbonnier, S., Caplier, A.: Using retina modelling to characterize blinking: comparison between EOG and video analysis. Mach. Vis. Appl. 23, 1195–1208 (2012)CrossRefGoogle Scholar
  2. 2.
    Akrout, B., Mahdi, W.: Spatio-temporal features for the automatic control of driver drowsiness state and lack of concentration. Mach. Vis. Appl. (MVA) 26, 1–13 (2015). SpringerCrossRefGoogle Scholar
  3. 3.
    Viola, P., Jones, M.: Robust real-time object detection. Int. J. Comput. Vis. 57, 137–154 (2001).
  4. 4.
    Teyeb, I., Jemai, O., Bouchrika, T., Ben Amar, C.: Detecting driver drowsiness using eyes recognition system based on wavelet network. In: 5th International Conference on Web and Information Technologies (ICWIT13) Proceedings, May 09–12, pp. 245–254, Hammamet, Tunisia (2013)Google Scholar
  5. 5.
    Jemai, O., Teyeb, I., Bouchrika, T., Ben Amar, C.: A novel approach for drowsy driver detection using eyes recognition system based on wavelet network. Int. J. Recent Contrib. Eng. (IJES) Sci. IT 1, 46–52 (2013)CrossRefGoogle Scholar
  6. 6.
    Teyeb, I., Jemai, O., Mourad, Z., Ben Amar, C.: A novel approach for drowsy driver detection using head posture estimation and eyes recognition system based on wavelet network. In: The Fifth International Conference on Information, Intelligence, Systems and Applications (IISA 2014) Proceedings, pp. 379-384, 07–09 July, Chania, Greece (2014). doi: 10.1109/IISA.2014.6878809
  7. 7.
    Teyeb, I., Jemai, O., Zaied, M., Ben Amar, C.: A drowsy driver detection system based on a new method of head posture estimation. In: Corchado, E., Lozano, J.A., Quintián, H., Yin, H. (eds.) IDEAL 2014. LNCS, vol. 8669, pp. 362–369. Springer, Cham (2014). doi: 10.1007/978-3-319-10840-7_44 Google Scholar
  8. 8.
    Teyeb, I., Jemai, O., Mourad, Z., Ben Amar, C.: A multi level system design for vigilance measurement based on head posture estimation and eyes blinking. In: Eighth International Conference on Machine Vision (ICMV 2015), Proceedings of SPIE, vol. 9875, 98751p, 8 December (2015). doi: 10.1117/12.2229616
  9. 9.
    Teyeb, I., Jemai, O., Mourad, Z., Ben Amar, C.: Vigilance measurement system through analysis of visual and emotional drivers signs using wavelet networks. In: Proceedings of the 15th International Conference on Intelligent Systems Design and Applications (ISDA 2015), pp. 140–147, 14–16 December, Marrakech, Maroc (2015)Google Scholar
  10. 10.
    Guedri, B., Zaied, M., Ben Amar, C.: Indexing and images retrieval by content. In: International Conference on High Performance Computing and Simulation (HPCS), 4–8 July, Istanbul, Turkey, pp. 369–37 (2011)Google Scholar
  11. 11.
    Ejbali, R., Zaied, M., Ben Amar, C.: Multi-input multi-output beta wavelet network: modeling of acoustic units for speech recognition. Int. J. Adv. Comput. Sci. Appl. (IJACSA) Sci. Inf. Organ. (SAI) 3(4), 38–44 (2012)Google Scholar
  12. 12.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems 25 (NIPS 2012), pp. 1097–1105 (2012)Google Scholar
  13. 13.
    Jemai, O., Ejbeli, R., Zaied, M., Ben Amar, C.: A speech recognition system based on hybrid wavelet network including a fuzzy decision support system. In: International Conference on Machine Vision (ICMV 2014), Proceedings of SPIE, 19–21 November, Milan, vol. 9445, pp. 944503-1–7, doi: 10.1117/12.2180554 (2015)
  14. 14.
    Sarbjit, S., Nikolaos, P.: Monitoring driver fatigue using facial analysis techniques. In: IEEE Conference on Intelligent Transportation Systems, Proceedings (ITSC), pp. 314–318, Tokyo, Japan (1999)Google Scholar
  15. 15.
    Horng, W., Chen, C., Chang, Y.: Driver fatigue detection based on eye tracking and dynamic template matching. In: IEEE International Conference on Networking, Sensing and Control, pp. 7–12, Taipei, Taiwan (2004)Google Scholar
  16. 16.
    Sharabaty, H., Jammes, B., Esteve, D.: EEG analysis using HHT : one step toward automatic drowsiness scoring. In: 2nd International Conference on Advanced Information Networking and Applications - Workshops (AINA Workshops 2008), pp. 826–831, Okinawa (2008)Google Scholar
  17. 17.
    Abtahi, S., Omidyeganeh, M., Shirmohammadi, S., Hariri, B.: A yawning detection dataset. In: Proceedings of ACM Multimedia Systems, pp. 24–28, 19–21 March, Singapore (2014)Google Scholar
  18. 18.
    Song, F., Tan, X., Liu, X., Chen, S.: Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients. Pattern Recogn. 47(9), 2825–2838 (2014)CrossRefGoogle Scholar
  19. 19.
    Celine, C., Abdullah, R., Mohamed, K., Fakhri, K.: A multi-modal driver fatigue and distration assessment system. Int. J. Intell. Transp. Syst. Res. 14, 1–22 (2015)Google Scholar
  20. 20.
    Liang, Y., Lee, J.D.: A hybrid Bayesian network approach to detect driver cognitive distraction transport. Res. Part C: Emerg. Technol. 38, 146–155 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Ahmed Snoun
    • 1
  • Ines Teyeb
    • 1
  • Olfa Jemai
    • 1
  • Mourad Zaied
    • 1
  1. 1.National Engineering School of Gabes (ENIG), RTIM: Research Team in Intelligent MachinesUniversity of GabesGabèsTunisia

Personalised recommendations