Skip to main content
SpringerLink
Log in

Multimodal focus attention and stress detection and feedback in an augmented driver simulator

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

This paper presents a driver simulator, which takes into account the information about the user’s state of mind (level of attention, fatigue state, stress state). The user’s state of mind analysis is based on video data and biological signals. Facial movements such as eyes blinking, yawning, head rotations, etc., are detected on video data: they are used in order to evaluate the fatigue and the attention level of the driver. The user’s electrocardiogram and galvanic skin response are recorded and analyzed in order to evaluate the stress level of the driver. A driver simulator software is modified so that the system is able to appropriately react to these critical situations of fatigue and stress: some audio and visual messages are sent to the driver, wheel vibrations are generated and the driver is supposed to react to the alert messages. A multi-threaded system is proposed to support multi-messages sent by the different modalities. Strategies for data fusion and fission are also provided. Some of these components are integrated within the first prototype of OpenInterface: the multimodal similar platform.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Benoit A, Caplier A (2005) Motion estimator inspired from biological model for head motion interpretation—WIAMIS05, Montreux, Suisse, April 2005

  2. Torralba AB, Herault J (1999) An efficient neuromorphic analog network for motion estimation. IEEE Trans Circuits Syst-I: Special issue on bio-inspired processors and CNNs for vision 46(2)

  3. Benoit A, Caplier A (2005) Head nods analysis : interpretation of non verbal communication gestures. IEEE ICIP, Genova

    Google Scholar 

  4. Benoit A, Caplier A (2005) Hypovigilence analysis: open or closed eye or mouth? Blinking or yawning frequency? In: IEEE AVSS, Como, Italy

  5. Machine Perception Toolbox (MPT). http://mplab.ucsd.edu/grants/project1/freesoftware/MPTWebSite/API/

  6. Bayes Net Toolbox for MatLab. http://www.cs.ubc.ca/∼murphyk/Software/BNT/bnt.html

  7. Qiang Ji, Zhiwei Zhu, Peilin Lan (2004) Real-time nonintrusive monitoring and prediction of driver fatigue. IEEE Trans Vehicular Technol 53(4):1052–1068

    Google Scholar 

  8. Force Feedback Direct Input Library. http://courses.washington.edu/css450/Fall2003/web_contents/direct_input_lib/DirectInput.html

  9. TORCS Driver Simulator. http://torcs.sourceforge.net/

  10. Similar Network of Excellence. http://www.similar.cc/

  11. Lal SK, Craig A (2002) Driver fatigue: electroencephalography and psychological assessment. Psychophysiology 39(3 May):313–321

    Google Scholar 

  12. Healey J, Seger J, Picard R Quantifying driver stress: developing a system for collecting and processing bio-metric signals in natural situation–MIT technical report. No. 483

  13. OpenCV library. http://www.intel.com/technology/computing/opencv

Download references

Acknowledgment

This work has been supported by the Simlar Network of Excellence.

Author information

Authors and Affiliations

  1. Gipsa-Lab/DIS, Grenoble, France

    Alexandre Benoit, Laurent Bonnaud, Alice Caplier & Phillipe Ngo

  2. Université Catholique de Louvain, Louvain-la-Neuve, Belgium

    Lionel Lawson & Daniela G. Trevisan

  3. Zagreb University, Zagreb, Croatia

    Vjekoslav Levacic

  4. Université Polytechnique de Mons, Mons, Belgium

    Céline Mancas

  5. University of Geneva, Geneva, Switzerland

    Guillaume Chanel

Authors
  1. Alexandre Benoit
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Laurent Bonnaud
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alice Caplier
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Phillipe Ngo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lionel Lawson
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Daniela G. Trevisan
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Vjekoslav Levacic
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Céline Mancas
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Guillaume Chanel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alice Caplier.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Benoit, A., Bonnaud, L., Caplier, A. et al. Multimodal focus attention and stress detection and feedback in an augmented driver simulator. Pers Ubiquit Comput 13, 33–41 (2009). https://doi.org/10.1007/s00779-007-0173-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-007-0173-0

Keywords

Search

Navigation