Assessing Cognitive State with Multiple Physiological Measures: A Modular Approach

  • Lee W. Sciarini
  • Denise Nicholson
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5638)


The purpose of this effort is to introduce a novel approach which can be used to determine how multiple minimally intrusive physiological sensors can be used together and validly applied to areas such as Augmented Cognition and Neuroergonomics. While researchers in these fields have established the utility of many physiological measures for informing when to adapt systems, the use of such measures together remains limited. Specifically, this effort will provide a contextual explanation of cognitive state, workload, and the measurement of both; provide a brief discussion on several relatively noninvasive physiological measures; explore what a modular cognitive state gauge should consist of; and finally, propose a framework based on the previous items that can be used to determine the interactions of the various measures in relation to the change of cognitive state.


Augmented Cognition Neuroergonomics Physiological Measures 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Reeves, L., Stanney, K., Axelsson, P., Young, P., Schmorrow, D.: Near-term, Mid-term, and Long-term Research Objectives for Augmented Cognition (a) Robust Controller Technology and (b) Mitigation Strategies. In: 4th International Conference on Augmented Cognition, pp. 282–289. Strategic Analysis, Inc., Arlington (2007)Google Scholar
  2. 2.
    Wickens, C.D.: Processing Resources in Attention. In: Parasuraman, R., Davies, D.R. (eds.) Varieties of Attention, pp. 63–102. Academic Press, London (1984)Google Scholar
  3. 3.
    Wickens, C.D.: Multiple Resources and Performance Prediction. Theoretical Issues in Ergonomics Science 3, 150–177 (2002)CrossRefGoogle Scholar
  4. 4.
    St John, M., Kobus, D.A., Morrison, J.G.: DARPA Augmented Cognition Technical Integration Experiment (TIE) Tech. Report 1905. United States Navy SPAWAR Systems Center, San Diego (2003)Google Scholar
  5. 5.
    St. John, M., Kobus, D.A., Morrison, J.G., Schmorrow, D.: Overview of the DARPA Augmented Cognition Technical Integration Experiment. Int. J. of HCI 17, 131–149 (2004)Google Scholar
  6. 6.
    Byrne, E.A., Parasuraman, R.: Psychophysiology and Adaptive Automation. Biological Psychology 42, 249–268 (1996)CrossRefPubMedGoogle Scholar
  7. 7.
    Kahneman, D.: Attention and Effort. Prentice-Hall, Englewood Cliffs (1973)Google Scholar
  8. 8.
    Brookhuis, K.A.: Psychophysiological Methods. In: Stanton, N., Hedge, A., Brookhuis, K., Salas, E., Hendrick, H. (eds.) Handbook of Human Factors and Ergonomics Methods, pp. 17-1–17-5. CRC Press, Boca Raton (2005)Google Scholar
  9. 9.
    Hadley, G., Guttman, J., Stringer, P.: Air traffic control specialist performance measurement database. (Tech. Report DOT/FAA/CT-TN99/17). Federal Aviation Administration William J. Hughes Technical Center, Atlantic City (1999)Google Scholar
  10. 10.
    Reinach, S.: Preliminary Development of a Railroad Dispatcher Taskload Assessment Tool: Identification of Dispatcher Tasks and Data Collection Methods. Tech. Report DOT/FRA/ORD-07/13. U.S. Department of Transportation Federal Railroad Administration Office of Research and Development, Washington (2007)Google Scholar
  11. 11.
    O’Donnell, R.D., Eggemeier, F.T.: Workload Assessment Methodology. In: Boff, K.R., Kaufman, L., Thomas, J.P. (eds.) Handbook of Perception and Human Performance. Cognitive Processes and Performance, vol. II, pp. 42-1–42-49. Wiley, New York (1986)Google Scholar
  12. 12.
    Wierwille, W.W., Eggemeier, F.T.: Recommendations for Mental Workload Measurements in a Test and Evaluation Environment. Human Factors 35, 263–281 (1993)Google Scholar
  13. 13.
    Wickens, C.D.: Workload and Situation Awareness. In: Hancock, P., Desmond, P. (eds.) Stress, Workload and Fatigue, pp. 443–450. Lawrence Erlbaum Associates, Mahwah (2001)Google Scholar
  14. 14.
    Paas, F., Juhani, E.T., Tabbers, H., Van Gerven, P.W.M.: Cognitive Load Measurement as a Means to Advance Cognitive Load Theory. Educational Psychologist 38, 63–71 (2003)CrossRefGoogle Scholar
  15. 15.
    Hart, S.G., Staveland, L.E.: Development of the NASA Task Load Index (TLX): Results of Experimental and Theoretical Research. In: Hancock, P., Meshkati, N. (eds.) Human Workload, pp. 138–183. North-Holland, Amsterdam (1988)Google Scholar
  16. 16.
    Scerbo, M.W., Freeman, F.G., Mikula, P.J., Parasuraman, R., Di Nocero, F., Prinzel III, L.J.: The Efficacy of Psychophysiological Measures for Implementing Adaptive Technology. Tech. Report NASA/TP-2001-211018. NASA Langley Research Center, Hampton (2001)Google Scholar
  17. 17.
    Morris, T.L., Miller, J.C.: Electooculographic and Performance Indices of Fatigue During Simulated Flight. Biological Psychology 42, 343–360 (1996)CrossRefPubMedGoogle Scholar
  18. 18.
    Veltman, J.A., Gaillard, A.W.K.: Physiological Indices of Workload in a Simulated Flight Task. Biological Psychology 42, 323–342 (1996)CrossRefPubMedGoogle Scholar
  19. 19.
    Backs, R.W., Ryan, A.M., Wilson, G.F.: Psychophysiological Measures During Continuous and Manual Performance. Human Factors 36, 514–531 (1994)PubMedGoogle Scholar
  20. 20.
    Hankins, T.C., Wilson, G.F.: A Comparison of Heart Rate, Eye Activity, EEG and Subjective Measures of Pilot Mental Workload During Flight. Aviation, Space & Environmental Medicine 69, 360–367 (1998)Google Scholar
  21. 21.
    Wilson, G.F.: An Analysis of Mental Workload in Pilots During Flight using Multiple Psychophysiological Measures. Int. Jnl. of Aviation Psych. 12, 3–18 (2002)CrossRefGoogle Scholar
  22. 22.
    Beatty, J.: Task-Evoked Pupillary Responses, Processing Load, and the Structure of Processing Resources. Psychological Bulletin 91, 276–292 (1982)CrossRefPubMedGoogle Scholar
  23. 23.
    Wilson, G.F., Eggemeier, F.T.: Psychophysiological Assessment of Workload in Multi-task Environments. In: Damos, D.L. (ed.) Multiple-task Performance, pp. 329–360. Taylor & Francis, London (1991)Google Scholar
  24. 24.
    Roscoe, A.H.: Heart Rate as Psychophysiological Measure for Inflight Workload Assessment. Ergonomics 36, 1055–1062 (1993)CrossRefPubMedGoogle Scholar
  25. 25.
    Wilson, G.F.: Applied Use of Cardiac and Respiration Measures: Practical Considerations and Precautions. Biological Psychology 34, 163–178 (1992)CrossRefPubMedGoogle Scholar
  26. 26.
    Mulder, L.J.M., De Waard, D., Brookhuis, K.A.: Estimating Mental Effort Using Heart Rate and Heart Rate Variability. In: Stanton, N., Hedge, A., Brookhuis, K., Salas, E., Hendrick, H. (eds.) Handbook of Human Factors and Ergonomics Methods, pp. 20-1–20-8. CRC Press, Boca Raton (2005)Google Scholar
  27. 27.
    Boucsein, W.: Electrodermal Measurement. In: Stanton, N., Hedge, A., Brookhuis, K., Salas, E., Hendrick, H. (eds.) Handbook of Human Factors and Ergonomics Methods, pp. 18-1–18-12. CRC Press, Boca Raton (2005)Google Scholar
  28. 28.
    Shi, Y., Ruiz, N., Taib, R., Choi, E.H.C., Chen, F.: Galvanic Skin Response (GSR) as an Index of Cognitive Load. In: CHI 2007, San Francisco (2007)Google Scholar
  29. 29.
    Akerstedt, T.: Ambulatory EEG Methods and Sleepiness. In: Stanton, N., Hedge, A., Brookhuis, K., Salas, E., Hendrick, H. (eds.) Handbook of Human Factors and Ergonomics Methods, pp. 21-1–21-7. CRC Press, Boca Raton (2005)Google Scholar
  30. 30.
    Berka, C., Levendowski, D.J., Lumicao, M.N., Yau, A., Davis, G., Zivkovic, V.T., Olmstead, R.E., Tremoulet, P.D., Craven, P.L.: EEG Correlates of Task Engagement and Mental Workload in Vigilance, Learning, and Memory Tasks. Aviation Space and Environmental Med. 78(Suppl. 5), B231–B244 (2007)Google Scholar
  31. 31.
    Gohara, T., Mizuta, H., Takeuchi, I., Tsuda, O., Yana, K., Yanai, T., Yamamoto, Y., Kishi, N.: Heart Rate Variability Change Induced by the Mental Stress: The Effect of Accumulated Fatigue. In: 15th Southern Biomed. Eng. Conf., pp. 367–369 (1996)Google Scholar
  32. 32.
    Reeves, L., Schomorrow, D.: Augmented cognition foundations and future directions - Enabling “anyone, anytime, anywhere” applications, pp. 263–272. Springer, Berlin (2007)Google Scholar
  33. 33.
    Vartak, A., Fidopiastis, C., Nicholson, D., Mikhael, W., Schmorrow, D.: Cognitive State Estimation for Adaptive Learning Systems using Wearable Physiological Sensors. In: 1st Intl Conf. on Biomedical Electronics and Devices (2008)Google Scholar
  34. 34.
    Sims, V., Mayer, R.: Domain Specificity of Spatial Expertise: The Case of Video Game Players. Applied Cognitive Psychology 16, 97–115 (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Lee W. Sciarini
    • 1
  • Denise Nicholson
    • 1
  1. 1.Applied Cognition and Training in Immersive Virtual Environments LaboratoryInstitute for Simulation and TrainingOrlandoUSA

Personalised recommendations