Real-Time Monitoring of Cognitive Workload of Airline Pilots in a Flight Simulator with fNIR Optical Brain Imaging Technology

  • Murat Perit ÇakırEmail author
  • Murat Vural
  • Süleyman Özgür Koç
  • Ahmet Toktaş
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9743)


Real-time monitoring of the flight crew’s health status with ambient and body sensors have become an important concern to improve the safety and the efficiency of flight operations. In this paper we report our preliminary findings on a functional near-infrared spectroscopy (fNIR) based online algorithm developed for real-time monitoring of mental workload of an airline pilot. We developed a linear discriminant analysis (LDA) based classifier that aims to predict low, moderate and high mental workload states based on a set of features computed over a moving window of oxy- and deoxy-hemoglobin measures obtained from 16 locations distributed over the prefrontal cortex. In this paper we explore the predictive power of a model trained for a single pilot over a sample of eight pilots and discuss the technical challenges involved with real-time measurement of brain activity in a flight simulator environment that involves other infra-red sources.


Mental workload estimation Optical brain imaging fNIR Neuroergonomics Linear discriminant analysis 


  1. 1.
    ICAO. Safety Report. International Civil Aviation Organization (2014). Accessed 29 Feb 2016
  2. 2.
    Borghini, G., Astolfi, L., Vecchiato, G., Mattia, D., Babiloni, F.: Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci. Biobehav. Rev. 44, 58–75 (2014)CrossRefGoogle Scholar
  3. 3.
    De Rivecourt, M., Kuperus, M.N., Post, W.J., Mulder, L.J.M.: Cardiovascular and eye activity measures as indices for momentary changes in mental effort during simulated flight. Ergonomics 51(9), 1295–1319 (2008)CrossRefGoogle Scholar
  4. 4.
    Dussault, C., Jouanin, J.C., Philippe, M., Guezennec, C.Y.: EEG and ECG changes during simulator operation reflect mental workload and vigilance. Aviat. Space Environ. Med. 76, 344–351 (2005)Google Scholar
  5. 5.
    Ayaz, H., Onaral, B., Izzetoglu, K., Shewokis, P.A., McKendrick, R., Parasuraman, R.: Continuous monitoring of brain dynamics with functional near infrared spectroscopy as a tool for neuroergonomic research: empirical examples and a technological development. Front. Human Neurosci. 7, 871 (2013)CrossRefGoogle Scholar
  6. 6.
    Gateau, T., Durantin, G., Lancelot, F., Scannella, S., Dehais, F.: Real-time state estimation in a flight simulator using fNIRS. PLoS ONE 10(3), e0121279 (2015)CrossRefGoogle Scholar
  7. 7.
    Herff, C., Heger, D., Fortmann, O., Hennrich, J., Putze, F., Schultz, T.: Mental workload during n-back task-quantified in the prefrontal cortex using fNIRS. Front. Human Neurosci. 7(1), 935–940 (2013)Google Scholar
  8. 8.
    Izzetoglu, M., Izzetoglu, K., Bunce, S., Ayaz, H., Devaraj, A., Onaral, B., Pourrezaei, K.: Functional near-infrared neuroimaging. IEEE Trans. Neural Syst. Rehabil. Eng. 13(2), 153–159 (2005)CrossRefGoogle Scholar
  9. 9.
    Obrig, H., Wenzel, R., Kohl, M., Horst, S., Wobst, P., Steinbrink, J., Villringer, A.: Near-infrared spectroscopy: does it function in functional activation studies of the adult brain? Int. J. Psychophysiol. 35(2), 125–142 (2000)CrossRefGoogle Scholar
  10. 10.
    Heeger, D.J., Ress, D.: What does fMRI tell us about neuronal activity? Nat. Rev. Neurosci. 3(2), 142–151 (2002)CrossRefGoogle Scholar
  11. 11.
    Wray, S., Cope, M., Delpy, D.T., Wyatt, J.S., Reynolds, E.O.R.: Characterization of the near infrared absorption spectra of cytochrome aa3 and haemoglobin for the non-invasive monitoring of cerebral oxygenation. Biochim. Biophys. Acta (BBA)-Bioenerg. 933(1), 184–192 (1988)CrossRefGoogle Scholar
  12. 12.
    Ayaz, H., Shewokis, P.A., Curtin, A., Izzetoglu, M., Izzetoglu, K., Onaral, B.: Using MazeSuite and functional near infrared spectroscopy to study learning in spatial navigation. J. Vis. Exp. 56(3443), 10–3791 (2011)Google Scholar
  13. 13.
    Ayaz, H.: Functional Near Infrared Spectroscopy based Brain Computer Interface. Ph.D. Thesis, Drexel University, Philadelphia, PA (2010)Google Scholar
  14. 14.
    Duncan, J.: The multiple-demand (MD) system of the primate brain: mental programs for intelligent behaviour. Trends Cogn. Sci. 14(4), 172–179 (2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Murat Perit Çakır
    • 1
    Email author
  • Murat Vural
    • 1
    • 2
  • Süleyman Özgür Koç
    • 2
  • Ahmet Toktaş
    • 2
  1. 1.Graduate School of InformaticsMiddle East Technical UniversityAnkaraTurkey
  2. 2.Turkish Aerospace IndustriesAnkaraTurkey

Personalised recommendations