Annals of Biomedical Engineering

, Volume 42, Issue 1, pp 162–176 | Cite as

Off-line and On-line Stress Detection Through Processing of the Pupil Diameter Signal

  • Peng Ren
  • Armando Barreto
  • Jian Huang
  • Ying Gao
  • Francisco R. Ortega
  • Malek Adjouadi
Article

Abstract

The pupil diameter (PD), controlled by the autonomic nervous system, seems to provide a strong indication of affective arousal, as found by previous research, but it has not been investigated fully yet. In this study, new approaches based on monitoring and processing the PD signal for off-line and on-line “relaxation” vs. “stress” differentiation are proposed. For the off-line approach, wavelet denoising, Kalman filtering, data normalization, and feature extraction are sequentially utilized. For the on-line approach, a hard threshold, a moving average window and three stress detection steps are implemented. In order to use only the most reliable data, two types of data selection methods (paired t test based on galvanic skin response (GSR) data and subject self-evaluation) are applied, achieving average classification accuracies up to 86.43 and 87.20% for off-line and 72.30 and 73.55% for on-line algorithms, with each set of selected data, respectively. The GSR was also monitored and processed in our experiments for comparison purposes, with the highest classification rate achieved being only 63.57% (based on the off-line processing algorithm). The overall results show that the PD signal is more effective and robust for differentiating “relaxation” vs. “stress,” in comparison with the traditionally used GSR signal.

Keywords

Autonomic nervous system (ANS) Pupil diameter (PD) Galvanic skin response (GSR) Wavelet denoising Kalman filtering Moving average window Backward differentiation Mathematical morphology 

Notes

Acknowledgments

This work was sponsored by NSF grants HRD-0833093 and CNS-0959985.

References

  1. 1.
    Andreassi, J. L. Psychophysiology: Human Behavior & Physiological Response. Mahwah, NJ: Lawrence Erlbaum Associates, 2007.Google Scholar
  2. 2.
    Andren, J., and P. Funk. A case-based approach using behavioural biometrics to determine a user’s stress level. In: ICCBR Workshops, 2005, pp. 9–17.Google Scholar
  3. 3.
    Beatty, J., and B. Lucero-Wagoner. Handbook of Psychophysiology. Cambridge: Cambridge University Press, 2000.Google Scholar
  4. 4.
    Begum, S., M. U. Ahmed, P. Funk, and N. Xiong. Using calibration and fuzzification of cases for improved diagnosis and treatment of stress. In: 8th European Conference on Case-Based Reasoning Workshop Proceedings, edited by M. Minor, 2006, pp. 113–122.Google Scholar
  5. 5.
    Cano-Vindel, A., J. J. Miguel-Tobal, H. Gonzalez-Ordi, and I. Iruarrizaga-Diez. Hyperventilation and anxiety experience. Anxiety Stress 13(2–3):291–302, 2007.Google Scholar
  6. 6.
    Fellous, M., and M. A. Arbib. Who Needs Emotions? The Brain Meets the Robot. New York: Oxford University Press, 2005.CrossRefGoogle Scholar
  7. 7.
    Field, A. Discovering Statistics Using SPSS (3rd ed.). New York: Sage, 2009.Google Scholar
  8. 8.
    Gao, Y., A. Barreto, and M. Adjouadi. Monitoring and Processing of the Pupil Diameter Signal for Affective Assessment of a Computer User. Lecture Notes in Computer Science (LNCS). LNCS 5610, 2009, pp. 49–58.Google Scholar
  9. 9.
    Gao, Y., A. Barreto, and M. Adjouadi. Affective assessment of a computer user through the processing of the pupil diameter signal. In: Innovations in Computing Sciences and Software Engineering, edited by T. Sobh, and K. Elleithy. New York: Springer, 2010, pp. 189–194.CrossRefGoogle Scholar
  10. 10.
    Granholm, E., and S. R. Steinhauer. Introduction: Pupillometric measures of cognitive and emotional processing. Int. J. Psychophysiol. 52:1–6, 2004.PubMedCrossRefGoogle Scholar
  11. 11.
    Grewal, M. S., and A. P. Andrews. Kalman Filtering: Theory and Practice Using MATLAB (3rd ed.). Hoboken, NJ: Wiley-IEEE Press, 2008.CrossRefGoogle Scholar
  12. 12.
    Healey, J. Wearable and Automotive Systems for Affect Recognition from Physiology. Ph.D. dissertation, MIT Media Lab, 2000.Google Scholar
  13. 13.
    Healey, J. A., and R. W. Picard. Detecting stress during real-world driving tasks using physiological sensors. IEEE Trans. Intell. Transp. Syst. 6(2):156–166, 2005.CrossRefGoogle Scholar
  14. 14.
    Hjemdahl, P., U. Freyschuss, A. Juhlin-Dannfelt, and B. Linde. Differentiated sympathetic activation during mental stress evoked by the Stroop test. Acta Physiol. Scand. Suppl. 527:25–29, 1984.PubMedGoogle Scholar
  15. 15.
    Holmqvist, K., M. Nystrom, R. Andersson, R. Dewhurst, H. Jarodzka, and J. Weijer. Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford: Oxford University Press, 2011.Google Scholar
  16. 16.
    Klingner, J. The pupillometric precision of a remote video eye tracker. In: Proceedings of the 2010 Symposium on Eye-Tracking Research Applications (ETRA ‘10). New York: ACM, 2010, pp. 259–262.Google Scholar
  17. 17.
    Lim, C. L., C. Rennie, R. J. Barry, H. Bahramali, I. Lazzaro, and B. Manor. Decomposing skin conductance into tonic and phasic components. Int. J. Psychophysiol. 24(2):97–109, 1997.CrossRefGoogle Scholar
  18. 18.
    Martini, F. H., and J. L. Nath. Fundamentals of Anatomy & Physiology (8th ed.). San Francisco: Benjamin Cummings, 2008.Google Scholar
  19. 19.
    Morgante, J. D., R. Zolfaghari, and S. P. Johnson. A critical test of temporal and spatial accuracy of the Tobii T60XL eye tracker. Infancy 17:9–32, 2012.CrossRefGoogle Scholar
  20. 20.
    Partala, T., and V. Surakka. Pupil size variation as an indication of affective processing. Int. J. Hum.–Comput. Stud. 59:185–198, 2003.CrossRefGoogle Scholar
  21. 21.
    Picard, W., and J. A. Healey. Wearable and automotive systems for affect recognition from physiology. Technical report, MIT, 2000.Google Scholar
  22. 22.
    Raymond, J. C. Dictionary of Psychology. New York: Routledge, 1999.Google Scholar
  23. 23.
    Shi, Y., N. Ruiz, R. Taib, E. Choi, and F. Chen. Galvanic skin response as an index of cognitive load. In: Proceeding of Computer–Human Interaction conference on Human Factors in Computing System, 2007, pp. 2651–2656.Google Scholar
  24. 24.
    Siegle, G. J., S. R. Steinhauer, and M. E. Thase. Pupillary assessment and computational modeling of early and sustained processing on the Stroop task in depression. Int. J. Psychophysiol. 52:63–76, 2004.PubMedCrossRefGoogle Scholar
  25. 25.
    Sierra, A. S., C. S. Ávila, J. G. Casanova, and G. B. Pozo. A stress-detection system based on physiological signals and fuzzy logic. IEEE Trans. Ind. Electron. 58(10):4857–4865, 2011.CrossRefGoogle Scholar
  26. 26.
    Sierra, A. S., C. S. Ávila, A. Mendaza-Ormaza, and J. G. Casanova. An approach to hand biometrics in mobile devices. SIViP 5(4):469–475, 2011.CrossRefGoogle Scholar
  27. 27.
    Smith, S. W. The Scientist and Engineer’s Guide to Digital Signal Processing. San Diego, CA: California Technical Publishing, 1997.Google Scholar
  28. 28.
    Steinhauer, S. R., G. J. Siegle, R. Condray, and M. Pless. Sympathetic and parasympathetic innervation of pupillary dilation during sustained processing. Int. J. Psychophysiol. 52:77–86, 2004.PubMedCrossRefGoogle Scholar
  29. 29.
    Stern, R. M., W. J. Ray, and K. S. Quigley. Psychophysiological Recording. Oxford: Oxford University Press, 2000.CrossRefGoogle Scholar
  30. 30.
    Stroop, J. R. Studies of interference in serial verbal reactions. J. Exp. Psychol. 18:643–662, 1935.CrossRefGoogle Scholar
  31. 31.
    Sun, F., C. Kuo, H. Cheng, S. Buthpitiya, P. Collins, and M. L. Griss. Activity-Aware Mental Stress Detection Using Physiological Sensors. Silicon Valley Campus, Paper 23, 2010.Google Scholar
  32. 32.
    Tognetti, S., M. Garbarino, M. Matteucci, and A. Bonarini. The affective triad: stimuli, questionnaires, and measurements. In: Proceeding of ACII’11, Vol. II, 2011, pp. 101–110.Google Scholar
  33. 33.
    Tourangeau, R., L. J. Rips, and K. A. Rasinski. The Psychology of Survey Response. Cambridge: Cambridge University Press, 2000.CrossRefGoogle Scholar
  34. 34.
    Tsai, S. S. Power Transformer Partial Discharge (PD) Acoustic Signal Detection Using Fiber Sensors and Wavelet Analysis, Modeling, and Simulation. Master’s Thesis, Virginia Polytechnic Institute and State University, Blacksburg, VA, 2002.Google Scholar
  35. 35.
    Tulen, J., H. P. Moleman, H. G. Steenis, and F. Boomsma. Characterization of stress reactions to the Stroop color word test. Pharmacol. Biochem. Behav. 32(1):9–15, 1989.PubMedCrossRefGoogle Scholar
  36. 36.
    University of California-Irvine. Short-term stress can affect learning and memory. ScienceDaily. March, 11, 2008.Google Scholar
  37. 37.
    Verney, S. P., E. Granholm, and S. P. Marshall. Pupillary responses on the visual backward masking task reflect general cognitive ability. Int. J. Psychophysiol. 52:23–36, 2004.PubMedCrossRefGoogle Scholar
  38. 38.
    Weigle, C., and D. C. Banks. Analysis of eye-tracking experiments performed on a Tobii T60. In: Proceedings of the SPIE6809, Visualization and Data Analysis, January, 2008.Google Scholar
  39. 39.
    Zhai, J., and A. Barreto. Stress detection in computer users through noninvasive monitoring of physiological signals. Biomed. Sci. Instrum. 42:495–500, 2006.PubMedGoogle Scholar

Copyright information

© Biomedical Engineering Society 2013

Authors and Affiliations

  • Peng Ren
    • 1
  • Armando Barreto
    • 2
  • Jian Huang
    • 2
  • Ying Gao
    • 3
  • Francisco R. Ortega
    • 4
  • Malek Adjouadi
    • 2
  1. 1.Biomedical Engineering DepartmentFlorida International UniversityMiamiUSA
  2. 2.Electrical and Computer Engineering DepartmentFlorida International UniversityMiamiUSA
  3. 3.Electrical Engineering DepartmentUniversity of Wisconsin-PlattevillePlattevilleUSA
  4. 4.School of Computing and Information SciencesFlorida International UniversityMiamiUSA

Personalised recommendations