Skip to main content
SpringerLink
Log in
Book cover

Advances in Neuroergonomics and Cognitive Engineering pp 145–156Cite as

Test-Retest Stability of EEG and Eye Tracking Metrics as Indicators of Variations in User State—An Analysis at a Group and an Individual Level

  • Conference paper
  • First Online:

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 488))

Abstract

EEG- and eye tracking metrics have been investigated for their potential as indicators of user state in a variety of studies. However, their stability over time has rarely been assessed and findings are reported predominantly on a group level. In this paper, we report a test-retest analysis that aimed to investigate—at group and individual levels—the temporal stability of fixation duration, pupil dilation, and two built-in metrics from the Emotiv EPOC EEG sensor, namely Engagement and Frustration. The retest confirmed the temporal stability of most physiological metrics at the group level. But analysis at an individual level revealed that outcomes differ strongly between and also within individuals from test to retest. The divergent results between individual and group level illustrate that group level findings are of limited value for applications such as adaptive systems requiring individual user state diagnosis.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    In accordance with Tomarken [6], we refer to the term stability for test-retest analyses across sessions and reliability for test-retest-analyses within sessions.

References

  1. Dirican, C., Göktürk, M.: Psychophysiological measures of human cognitive states applied in human computer interaction. Procedia Comput. Sci. 3, 1361–1367 (2011)

    Article  Google Scholar 

  2. Kruse, A.A., Schmorrow, D.D.: Session Overview: foundations of augmented cognition. In Proceedings of the 11th International Conference on Human-Computer Interaction (Augmented Cognition International) (2005)

    Google Scholar 

  3. Berka, C., Levendowski, D.J., Davis, G., Whitmoyer, M., Hale, K., Fuchs, S.: Objective measures of situational awareness using neurophysiology technology. In: Schmorrow, D.D., Stanney, K.M., Reeves, L.M. (eds.) Foundations of Augmented Cognition, pp. 145–154. Strategic Analysis Inc, Arlington, VA (2006)

    Google Scholar 

  4. Schwarz, J., Fuchs, S., Flemisch, F.: Towards a more holistic view on user state assessment in adaptive human-computer interaction. In: IEEE International Conference on Systems, Man, and Cybernetics (SMC 2014), pp. 1228–1234 (2014)

    Google Scholar 

  5. Faulstich, M.E., Williamson, D.A., McKenzie, S.J., Duchmann, E.G., Hutchinson, K.M., Blouin, D.C.: Temporal stability of psychophysiological responding: a comparative analysis of mental and physical stressors. Int. J. Neurosci. 30(1–2), 65–72 (1986)

    Article  Google Scholar 

  6. Tomarken, A.J.: A psychometric perspective on psychophysiological measures. Psychol. Assess. 7(3), 387–395 (1995)

    Article  Google Scholar 

  7. Schwarz, J., Fuchs, S.: Efficacy of metrics from a low-cost EEG sensor for multi-dimensional user state assessment. In: Wienrich, C., Zander, T. Gramann, K. (eds.) Trends in Neuroergonomics: Proceedings of the 11. Berlin Workshop Human-Machine Systems, pp. 20–21. Universitätsverlag der TU Berlin, Berlin (2015)

    Google Scholar 

  8. Neerincx, M.A.: Cognitive task load design: model, methods and examples. In: Hollnagel, E. (ed.) Handbook of Cognitive Task Design, pp. 283–305. Lawrence Erlbaum Associates, Mahwah, NJ (2003)

    Chapter  Google Scholar 

  9. Reuderink, B., Nijholt, A., Poel, M.: Affective pacman: a frustrating game for brain-computer interface experiments. In: Third International Conference on Intelligent Technologies for Interactive Entertainment, INTETAIN 2009, pp. 221—227. Amsterdam, The Netherlands (2009)

    Google Scholar 

  10. Diener, H., Oertel, K.: Experimental approach to affective interaction in games. In: Pan, Z., et al. (eds.) Edutainment 2006, LNCS, vol. 3942, pp. 507–518. Springer, Heidelberg (2006)

    Google Scholar 

  11. Pomplun, M., Reingold, E.M., Shen, J.: Investigating the visual span in comparative search: the effects of task difficulty and divided attention. Cognition 81(2), B57–B67 (2001)

    Article  Google Scholar 

  12. King, L.A.: Visual navigation patterns and cognitive load. In: Foundations of augmented cognition. Neuroergonomics and operational neuroscience, pp. 254–259. Springer, Berlin (2009)

    Google Scholar 

  13. Meghanathan, R. N., van Leeuwen, C., Nikolaev, A. R.: Fixation duration surpasses pupil size as a measure of memory load in free viewing. Front. Hum. Neurosci. 8, 1063 http://doi.org/10.3389/fnhum.2014.01063 (2014)

  14. Duchowski, A.T.: A breadth-first survey of eye tracking applications. Behav. Res. Methods Instrum. Comput. 34, 455–471 (2002)

    Article  Google Scholar 

  15. De Rivecourt, M., Kuperus, M.N., Post, W.J., Mulder, L.J.M.: Cardiovascular and eye activity measures as indices for momentary changes in mental effort during simulated flight. Ergonomics 51(9), 1295–1319 (2008)

    Article  Google Scholar 

  16. Richer, F., Beatty, J.: Pupillary dilations in movement preparation and execution. Psychophysiology 22(2), 204–207 (1985)

    Article  Google Scholar 

  17. Van Orden, K.F., Limbert, W., Makeig, S., Jung, T.P.: Eye activity correlates of workload during a visuospatial memory task. Hum. Factors 43(1), 111–121 (2001)

    Article  Google Scholar 

  18. Iqbal, S.T., Zheng, X.S., Bailey, B.P.: Task-evoked pupillary response to mental workload in human computer interaction. In: Ext. Abstracts CHI 2004, pp. 1477–1480. ACM Press (2004)

    Google Scholar 

  19. Jiang, X., Zheng, B., Bednarik, R., Atkins, M.S.: Pupil responses to continuous aiming movements. Int. J. Hum Comput. Stud. 83, 1–11 (2015)

    Article  Google Scholar 

  20. Kramer, A.F.: Physiological metrics of mental workload: a review of recent progress. In: Damos, D. (ed) Multiple Task Performance, pp. 279—328. Taylor and Francis, London (1991)

    Google Scholar 

  21. Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. In: Hancock, P.A., Meshkati, N. (eds.) Human Mental Workload, pp. 139–183. NorthHolland, Amsterdam (1988)

    Chapter  Google Scholar 

  22. Bradley, M.M.: Lang. P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49–59 (1994)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Fraunhofer FKIE, Fraunhoferstr. 20, 53343, Wachtberg, Germany

    Jessica Schwarz & Sven Fuchs

Authors
  1. Jessica Schwarz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sven Fuchs
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jessica Schwarz .

Editor information

Editors and Affiliations

  1. Design Interactive, Inc. , Orlando, USA

    Kelly S. Hale

  2. Design Interactive, Inc. , Orlando, USA

    Kay M. Stanney

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing Switzerland

About this paper

Cite this paper

Schwarz, J., Fuchs, S. (2017). Test-Retest Stability of EEG and Eye Tracking Metrics as Indicators of Variations in User State—An Analysis at a Group and an Individual Level. In: Hale, K., Stanney, K. (eds) Advances in Neuroergonomics and Cognitive Engineering. Advances in Intelligent Systems and Computing, vol 488. Springer, Cham. https://doi.org/10.1007/978-3-319-41691-5_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-41691-5_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-41690-8

  • Online ISBN: 978-3-319-41691-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation