Abstract
Deployment of portable neuroimaging technologies to operating settings could help assess cognitive states of personnel assigned to perform critical tasks and thus help improve efficiency and safety of human machine systems. Functional Near Infrared Spectroscopy (fNIR) is an emerging noninvasive brain imaging technology that relies on optical techniques to detect brain hemodynamics within the prefrontal cortex in response to sensory, motor, or cognitive activation. Collaborating with the FAA William J. Hughes Technical Center, fNIR has been used to monitor twenty four certified professional controllers as they manage realistic Air Traffic Control (ATC) scenarios under typical and emergent conditions. We have implemented a normalization procedure to estimate cognitive workload levels from fNIR signals during ATC by developing linear regression models that were informed by the respective participants’ prior n-back data. This normalization can account for oxygenation variance due to inter-personal physiological differences. Results indicate that fNIR is sensitive task loads during ATC.
Chapter PDF
Similar content being viewed by others
Keywords
References
FAA: FAA’s NextGen Implementation Plan. NextGen Integration and Implementation Office (2010)
Schmorrow, D., Kruse, A.A.: DARPA’s Augmented Cognition Program-tomorrow’s human computer interaction from vision to reality: building cognitively aware computational systems. In: Proceedings of the IEEE 7th Human Factors and Power Plants, p. 7, 1-4. IEEE, Scottsdale (2002)
Reeves, L., Schmorrow, D., Stanney, K.: Augmented Cognition and Cognitive State Assessment Technology – Near-Term, Mid-Term, and Long-Term Research Objectives. In: Schmorrow, D.D., Reeves, L.M. (eds.) HCII 2007 and FAC 2007. LNCS (LNAI), vol. 4565, pp. 220–228. Springer, Heidelberg (2007)
Parasuraman, R., Wilson, G.: Putting the brain to work: Neuroergonomics past, present, and future. Human factors 50, 468 (2008)
Cabeza, R., Nyberg, L.: Imaging cognition II: An empirical review of 275 PET and fMRI studies. Journal of Cognitive Neuroscience 12, 1–47 (2000)
Wood, J., Grafman, J.: Human prefrontal cortex: processing and representational perspectives. Nature Reviews Neuroscience 4, 139–147 (2003)
Ramnani, N., Owen, A.: Anterior prefrontal cortex: insights into function from anatomy and neuroimaging. Nature Reviews Neuroscience 5, 184–194 (2004)
Osaka, N., Osaka, M., Kondo, H., Morishita, M., Fukuyama, H., Shibasaki, H.: The neural basis of executive function in working memory: an fMRI study based on individual differences. Neuroimage 21, 623–631 (2004)
Chance, B., Zhuang, Z., UnAh, C., Alter, C., Lipton, L.: Cognition-activated low-frequency modulation of light absorption in human brain. Proceedings of the National Academy of Sciences of the United States of America 90, 3770–3774 (1993)
Villringer, A., Chance, B.: Non-invasive optical spectroscopy and imaging of human brain function. Trends in neurosciences 20, 435–442 (1997)
Strangman, G., Boas, D.A., Sutton, J.P.: Non-invasive neuroimaging using near-infrared light. Biological psychiatry 52, 679–693 (2002)
Izzetoglu, K., Bunce, S., Onaral, B., Pourrezaei, K., Chance, B.: Functional Optical Brain Imaging Using Near-Infrared During Cognitive Tasks. International Journal of Human-Computer Interaction 17, 211–227 (2004)
Izzetoglu, M., Izzetoglu, K., Bunce, S., Ayaz, H., Devaraj, A., Onaral, B., Pourrezaei, K.: Functional near-infrared neuroimaging. IEEE Trans Neural Syst Rehabil Eng 13, 153–159 (2005)
Ayaz, H., Willems, B., Bunce, B., Shewokis, P.A., Izzetoglu, K., Hah, S., Deshmukh, A., Onaral, B.: Cognitive Workload Assessment of Air Traffic Controllers Using Optical Brain Imaging Sensors. In: Marek, T., Karwowski, W., Rice, V. (eds.) Advances in Understanding Human Performance: Neuroergonomics, Human Factors Design, and Special Populations, pp. 21–31. CRC Press, Boca Raton (2010)
Menda, J., Hing, J.T., Ayaz, H., Shewokis, P.A., Izzetoglu, K., Onaral, B., Oh, P.: Optical Brain Imaging to Enhance UAV Operator Training, Evaluation, and Interface Development. Journal of Intelligent & Robotic Systems 61, 423–443 (2010)
Willems, B., Hah, S., Phillips, R.: The effect of data link on en route controller workload. FAA William J. Hughes Technical Center (2006)
Hah, S., Willems, B., Phillips, R.: The effect of air traffic increase on controller workload. Human Factors and Ergonomics Society Annual Meeting 50, 50–54 (2006)
Cain, B.: A Review of the Mental Workload Literature. Defence Research and Development, Toronto, Canada, Published online (2007), http://handle.dtic.mil/100.2/ADA474193
Welford, A.: Forty years of experimental psychology in relation to age: retrospect and prospect. Experimental gerontology 21, 469–481 (1986)
Hancock, P.A., Chignell, M.H.: Mental workload dynamics in adaptive interface design. IEEE Transactions on Systems, Man and Cybernetics 18, 647–658 (1988)
Cooper, G., Harper, R.: The use of pilot rating in the evaluation of aircraft handling qualities. NASA, Washington (1969)
Sheridan, T., Simpson, R.: Toward the definition and measurement of the mental workload of transport pilots (FTL Rept. R 79-4). Cambridge, MA: Massachusetts Institute of Technology, Flight Transportation Laboratory (1979)
Hart, S., Staveland, L.: Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Human mental workload 1, 139–183 (1988)
Paas, F.G.W.C., Van Merriënboer, J.J.G.: The efficiency of instructional conditions: An approach to combine mental effort and performance measures. Human Factors: The Journal of the Human Factors and Ergonomics Society 35, 737–743 (1993)
Embrey, D., Blackett, C., Marsden, P., Peachey, M.: Development of a Human Cognitive Workload Assessment Tool. MCA Final Report, Lancashire (2006)
Meshkati, N., Hancock, P.A., Rahimi, M., Dawes, S.M.: Techniques of mental workload assessment. In: Wilson, J., Corlett, E.N. (eds.) Evaluation of human work: A practical ergonomics methodology, pp. 749–782. Taylor & Francis, London (1995)
Ahlstrom, U., Friedman-Berg, F.J.: Using eye movement activity as a correlate of cognitive workload. International Journal of Industrial Ergonomics 36, 623–636 (2006)
Veltman, J., Gaillard, A.: Physiological indices of workload in a simulated flight task. Biological Psychology 42, 323–342 (1996)
Neumann, D.L., Lipp, O.V.: Spontaneous and reflexive eye activity measures of mental workload. Australian Journal of Psychology 54, 174–179 (2002)
Wang, L.-m., Duffy, V.G., Du, Y.: A Composite Measure for the Evaluation of Mental Workload. In: Duffy, V.G. (ed.) HCII 2007 and DHM 2007. LNCS, vol. 4561, pp. 460–466. Springer, Heidelberg (2007)
Helander, M.: Applicability of drivers’ electrodermal response to the design of the traffic environment. Journal of Applied Psychology 63, 481–488 (1978)
Bedny, G., Karwowski, W., Seglin, M.: A heart rate evaluation approach to determine cost-effectiveness an ergonomics intervention. International Journal of Occupational Safety and Ergonomics: JOSE 7, 121–133 (2001)
Roscoe, A.: Assessing pilot workload: Why measure heart rate, HRV and respiration? Biological Psychology 34, 259–287 (1992)
Brookings, J., Wilson, G., Swain, C.: Psychophysiological responses to changes in workload during simulated air traffic control. Biological Psychology 42, 361–377 (1996)
Berka, C., Levendowski, D.J., Lumicao, M.N., Yau, A., Davis, G., Zivkovic, V.T., Olmstead, R.E., Tremoulet, P.D., Craven, P.L.: EEG Correlates of Task Engagement and Mental Workload in Vigilance, Learning, and Memory Tasks. Aviation, space, and environmental medicine 78, B231–B244 (2007)
Bunce, S.C., Izzetoglu, M., Izzetoglu, K., Onaral, B., Pourrezaei, K.: Functional near-infrared spectroscopy: An Emerging Neuroimaging Modality. IEEE Eng Med Biol Mag 25, 54–62 (2006)
Ayaz, H., Shewokis, P., Bunce, S., Schultheis, M., Onaral, B.: Assessment of Cognitive Neural Correlates for a Functional Near Infrared-Based Brain Computer Interface System. In: Schmorrow, D., Estabrooke, I., Grootjen, M. (eds.) FAC 2009. LNCS, vol. 5638, pp. 699–708. Springer, Heidelberg (2009)
Ayaz, H., Izzetoglu, M., Platek, S.M., Bunce, S., Izzetoglu, K., Pourrezaei, K., Onaral, B.: Registering fNIR data to brain surface image using MRI templates. In: Conf Proc IEEE Eng Med Biol Soc, pp. 2671–2674 (2006)
Bechara, A., Damasio, H., Tranel, D., Anderson, S.: Dissociation of working memory from decision making within the human prefrontal cortex. Journal of Neuroscience 18, 428–437 (1998)
Koechlin, E., Corrado, G., Pietrini, P., Grafman, J.: Dissociating the role of the medial and lateral anterior prefrontal cortex in human planning. Proceedings of the National Academy of Sciences of the United States of America 97, 7651–7656 (2000)
Simons, J., Gilbert, S., Owen, A., Fletcher, P., Burgess, P.: Distinct roles for lateral and medial anterior prefrontal cortex in contextual recollection. Journal of Neurophysiology 94, 813–820 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ayaz, H. et al. (2011). Estimation of Cognitive Workload during Simulated Air Traffic Control Using Optical Brain Imaging Sensors. In: Schmorrow, D.D., Fidopiastis, C.M. (eds) Foundations of Augmented Cognition. Directing the Future of Adaptive Systems. FAC 2011. Lecture Notes in Computer Science(), vol 6780. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21852-1_63
Download citation
DOI: https://doi.org/10.1007/978-3-642-21852-1_63
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21851-4
Online ISBN: 978-3-642-21852-1
eBook Packages: Computer ScienceComputer Science (R0)