Advertisement

Artificial Life and Robotics

, Volume 23, Issue 3, pp 367–372 | Cite as

Estimating autonomic nerve activity using variance of thermal face images

  • Shogo MatsunoEmail author
  • Tota Mizuno
  • Hirotoshi Asano
  • Kazuyuki Mito
  • Naoaki Itakura
Original Article

Abstract

In this paper, we propose a novel method for evaluating mental workload (MWL) using variances in facial temperature. Moreover, our method aims to evaluate autonomic nerve activity using single facial thermal imaging. The autonomic nervous system is active under MWL. In previous studies, temperature differences between the nasal and forehead portions of the face were used in MWL evaluation and estimation. Hence, nasal skin temperature (NST) is said to be a reliable indicator of autonomic nerve activity. In addition, autonomic nerve activity has little effect on forehead temperature; thus, temperature differences between the nasal and forehead portions of the face have also been demonstrated to be a good indicator of autonomic nerve activity (along with other physiological indicators such as EEG and heart rate). However, these approaches have not considered temperature changes in other parts of the face. Thus, we propose novel method using variances in temperature for the entire face. Our proposed method enables capture of other parts of the face for temperature monitoring, thereby increasing evaluation and estimation accuracy at higher sensitivity levels than conventional methods. Finally, we also examined whether further high-precision evaluation and estimation was feasible. Our results proved that our proposed method is a highly accurate evaluation method compared with results obtained in previous studies using NST.

Keywords

Facial thermal image Nasal skin temperature Mental workload Health monitoring 

References

  1. 1.
    Haga S (2001), Theory and measurement of mental work-load. Japan Publication Service, TokyoGoogle Scholar
  2. 2.
    Murata A (1992) Measurement of mental workload by heart rate variability indexes. Jpn Ergon Soc 28(2):91–99MathSciNetCrossRefGoogle Scholar
  3. 3.
    Hirayanagi K, Iwasaki K, Kanda S, Yajima K (1996) An experimental study on the measurement and assessment of mental workload (MWL). Jpn Ergon Soc 32(5):251–259CrossRefGoogle Scholar
  4. 4.
    Morgan JF, Hancock PA (2011) The effect of prior task loading on mental workload: an example of hysteresis in driving. Hum Factors 53(1):75–86CrossRefGoogle Scholar
  5. 5.
    Tokunaga R, Hagiwara T, Kagaya S, Onodera Y (2000) Effects of talking through cellular telephone on driving behavior. Jpn Soc Civ Eng 17:995–1000Google Scholar
  6. 6.
    Matsunaga H, Nakazawa H (2000) Relation between mental workload and satisfaction measures evaluated from EEG. Jpn Soc Mech Eng 66(648):2884–2890Google Scholar
  7. 7.
    Karashima M, Yamazaki T (2003) A study on the relationship between the time pressure by time constraints and the time estimations. Jpn Ergon Soc 39(1):30–37CrossRefGoogle Scholar
  8. 8.
    Zenju H, Nozawa A, Tanaka H, Ide H (2004) The estimation of unpleasant and pleasant states by nasal thermogram, Institute of Electrical Engineers of Japan. Trans Electron Inf Syst 124(1):213–214Google Scholar
  9. 9.
    Kaname H, Kenichi I, Shougo K, Kazuyoshi Y (1996) An experimental study on the measurement and assessment of mental workload (MWL). Jpn Ergon Soc 32(5):251–259CrossRefGoogle Scholar
  10. 10.
    Hiro I (1988) Quantitative evaluation of mental work by thermography. Soc Instrum Control Eng 24(2):107–111Google Scholar
  11. 11.
    Mitsuhiko K, Tomoyuki Y (2003) A study on the relationship between the time pressure by time constraints and the time estimations. Jpn Ergon Soc 39(1):30–37CrossRefGoogle Scholar
  12. 12.
    Hioki K, Nozawa A, Mizuno T, Ide H (2007) Evaluation of the effect of intermittent mental work-load by nasal skin temperature, Institute of Electrical Engineers of Japan. Trans Electron Inf Syst 127(7):1000–1006Google Scholar
  13. 13.
    Nozawa A, Mizuno T, Ide H, Asano H, Nomura S (2010), Evaluation of the effect of intermittent mental work-load by nasal skin temperature. IEICE Trans Inf Syst J93-D(4):535–543Google Scholar
  14. 14.
    Nozawa A, Tomono S, Mizuno T, Ide H (2006) Detection of fight or flight reaction on facial skin thermogram using spatio-temporal spectrum differential analysis. IEEJ Trans Fundam Mater 126(6):470–477CrossRefGoogle Scholar
  15. 15.
    Sakamoto R, Nozawa A, Tanaka H, Mizuno T, Ide H (2006) Evaluation of the driver’s temporary arousal level by facial skin thermogram—effect of surrounding temperature and wind on the thermogram. IEEJ Trans Electron Inf Syst 126(7):804–809Google Scholar
  16. 16.
    Mizuno T, Nomura S, Nozawa A, Asano H, Ide H (2010), Evaluation of the effect of intermittent mental work-load by nasal skin temperature. IEICE Trans Inf Syst J93-D(4):535–543Google Scholar
  17. 17.
    Mizuno T, Nakategawa N, Kume Y (2012) Color influences on human beings evaluated by nasal skin temperature. Artif Life Robot 16(4):519–522CrossRefGoogle Scholar
  18. 18.
    Jarlier S, Grandjean D, Delplanque S, N’Diaye K, Cayeux I, Velazco M, Sander D, Vuilleumier P, Scherer KR (2011) Thermal analysis of facial muscles contractions. IEEE Trans Affect Comput 2(1):2–9CrossRefGoogle Scholar
  19. 19.
    Nhan BR, Chau T (2010) Classifying affective states using thermal infrared imaging of the human face. IEEE Trans Biomed Eng 57(4):979–987CrossRefGoogle Scholar
  20. 20.
    Kawazura S, Asano H, Nozawa A, Uchida M, Itakura N, Mito K, Mizuno T (2014) Estimation of the autonomic nerve activity with facial thermogram. In: Proceedings of the 11th international conference on electrical engineering/electronics, computer, telecommunications and information technology (ECTI-CON), Thailand, May 14–17, 2014 (CD-ROM), vol 1320, pp 1–6Google Scholar
  21. 21.
    Mizuno T, Kawazura S, Asano H, Mito K, Kume Y, Itakura N (2015) Evaluation of the autonomic nerve activity using facial thermal image. In: Proceedings of the international symposium on artificial life and robotics (AROB 20th), Japan, 2015, pp 512–514Google Scholar
  22. 22.
    Mizino T, Sakai T, Kawazura S, Asano H, Akehi K, Matsuno S, Mito K, Kume Y, Itakura N (2015) Facial skin temperature fluctuation by mental work-load with thermography. In: Proceedings of the international conference on electronics and software science (ICESS), Japan, July 2015, pp 212–215Google Scholar
  23. 23.
    Mizuno T, Sakai T, Kawazura S, Asano H, Akehi K, Matsuno S, Mito K, Kume Y, Itakura N (2016) Measuring facial skin temperature fluctuation caused by mental work-load with infrared thermography. Trans Electron Inf Syst 136(11):1581–1585Google Scholar
  24. 24.
    Watanabe S, Matsumoto Y (2011) An analysis of information skills for students based on a visual analog scale. J Biomed Fuzzy Syst Assoc 13(1):57–62Google Scholar
  25. 25.
    Akabayashi A, Yokoyama K, Araki S, Shimada K (1991) Clinical application of the Japanese edition of profile of mood states (POMS). Jpn J Psychosom Med 31(7):577–582Google Scholar

Copyright information

© ISAROB 2018

Authors and Affiliations

  • Shogo Matsuno
    • 1
    • 4
    Email author
  • Tota Mizuno
    • 2
  • Hirotoshi Asano
    • 3
  • Kazuyuki Mito
    • 2
  • Naoaki Itakura
    • 2
  1. 1.Toyohashi University of TechnologyToyohashiJapan
  2. 2.The University of Electro-CommunicationsChofuJapan
  3. 3.Kagawa UniversityTakamatsuJapan
  4. 4.Hotto Link Inc.ChiyodaJapan

Personalised recommendations