Skip to main content

Dangerous State Detection in Vehicle Cabin Based on Audiovisual Analysis with Smartphone Sensors

Part of the Advances in Intelligent Systems and Computing book series (AISC,volume 1250)


The paper presents the context-based approach for monitoring in-vehicle driver behavior based on the audiovisual analysis with aid of smartphone sensors, essentially utilizing front-facing camera and microphone. We propose the approach of driver monitoring system focused on recognizing situations whether the driver is drowsy or distracted, and reducing traffic accidents rate by generating context-relevant recommendations and perceiving driver’s feedback in a form of requested audio response to certain speech commands given by the smartphone. We efficiently utilize the information about driving behavior and the context to make sure that the driver actually followed the given recommendations that in the result will aid to reduce the probability of traffic accident. For example, audio signal produced by the smartphone’s microphone is used to check whether the driver increased or decreased the music volume inside the vehicle cabin. If the driver did not proceed with the recommendations, the driver is prompted to response with the voice command, and in this way, to confirm its alertness to current driving situation.


  • Driver
  • Dangerous state
  • Audio-based assistance
  • Context
  • Vehicle

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


  1. 2018 road safety statistics: what is behind the figures? Accessed 14 Dec 2019

  2. The EU Wants Cars To Have Speed Limiters and More by 2022. Accessed 14 Dec 2019

  3. Volvo Vision 2020. Accessed 14 Dec 2019

  4. Kashevnik, A., Lashkov I.: Decision support system for drivers passengers: smartphone-based reference model and evaluation. In: Conference of Open Innovation Association, FRUCT, pp. 166–171. IEEE Computer Society (2018)

    Google Scholar 

  5. Kashevnik, A., Lashkov, I., Gurtov, A.: Methodology and mobile application for driver behavior analysis and accident prevention. IEEE Trans. Intell. Transp. Syst. 21(6), 1–10 (2019)

    CrossRef  Google Scholar 

  6. Harrison, Y., Horne, J.A.: Sleep deprivation affects speech. J. Sleep Res. Sleep Med. 20(10), 871–877 (1997)

    Google Scholar 

  7. Krajewski, J., Batliner, A., Golz, M.: Acoustic sleepiness detection: framework and validation of a speech-adapted pattern recognition approach. Behav. Res. Methods 41, 795–804 (2009)

    CrossRef  Google Scholar 

  8. Batliner, A., Steidl, S., Nöth, E.: Releasing a thoroughly annotated and processed spontaneous emotional database: the FAU Aibo Emotion Corpus (2008)

    Google Scholar 

  9. Martin, V. P., Rouas, J.-L., Thivel, P., Franchi J.-A. M., Philip, P.: Towards automatic sleepiness measurement through speech, pp. 1–10 (2019)

    Google Scholar 

  10. Shahid, A., Wilkinson, K., Marcu, S., Shapiro, C.M.: Karolinska sleepiness scale (KSS). Psychology (2011)

    Google Scholar 

  11. Fraile, R., Godinol lorente, J.: Cepstral peak prominence: a comprehensive analysis. Biomed. Sig. Process. Control. 14(1), 42–54 (2014)

    CrossRef  Google Scholar 

  12. Fernandes, J., Teixeira, F., Guedes, V., Junior, A., Teixeira, J.: Harmonic to noise ratio measurement - selection of window and length. Proc. Comput. Sci. 138, 280–285 (2018)

    CrossRef  Google Scholar 

  13. Prokhorov, D., Kalik S., Varri C.: Toyota motor corp. system and method for reducing boredom while driving. US7982620B2, United States Patent and Trademark Office, 19 June 2011

    Google Scholar 

  14. Martin, V.P., Rouas, J., Thivel, P., Krajewski, J.: Sleepiness detection on read speech using simple features. In: 2019 International Conference on Speech Technology and Human-Computer Dialogue (SpeD), Timisoara, Romania, pp. 1–7 (2019)

    Google Scholar 

  15. Kamaruddin, N., Wahab, A.: Heterogeneous driver behavior state recognition using speech signal. In: Proceedings of the 10th WSEAS International Conference on Power Systems and Systems Science, pp. 207–212 (2011)

    Google Scholar 

  16. Angkititrakul, P., Kwak, D., Choi, S., Kim, J., PhucPhan, A., Sathyanarayana, A., Hansen, J. H. L.: Getting start with UTDrive: driver-behavior modeling and assessment of distraction for in-vehicle speech systems. In: INTERSPEECH-2007 – 8th Annual Conference of the International Speech Communication Association, Belgium, pp. 1334–1337 (2007)

    Google Scholar 

  17. Sommer, D., Golz, M.: Evaluation of PERCLOS based current fatigue monitoring technologies. In: 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, Buenos Aires, pp. 4456–4459 (2010)

    Google Scholar 

  18. Google Play – Drive Safely. Accessed 14 Dec 2019

Download references


Reference model of the driver monitoring system has been developed in scope of Russian Foundation for Basic Research project # 17-29-07073. Audiovisual approach for dangerous state determination is supported by the Russian Foundation for Basic Research project # 19-29-09081. Implementation has been done in scope of Russian State Research # 0073-2019-0005.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Igor Lashkov .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lashkov, I., Kashevnik, A., Shilov, N. (2021). Dangerous State Detection in Vehicle Cabin Based on Audiovisual Analysis with Smartphone Sensors. In: Arai, K., Kapoor, S., Bhatia, R. (eds) Intelligent Systems and Applications. IntelliSys 2020. Advances in Intelligent Systems and Computing, vol 1250. Springer, Cham.

Download citation