Skip to main content

Suitability of Physiological, Self-report and Behavioral Measures for Assessing Mental Workload in Pilots

  • Conference paper
  • First Online:
Engineering Psychology and Cognitive Ergonomics (HCII 2023)

Abstract

Adaptive automation shall support users in a flexible way. One way to achieve this could be by monitoring cognitive states of pilots in order to anticipate an individual’s need for support. A special challenge lies in choosing methods that enable a valid measurement of the cognitive state in question since different measures are associated with distinct strengths and weaknesses. For example, practical considerations like environmental factors, wearing comfort and intrusiveness have to be considered. The objective of this paper is to provide a collection of physiological, self-report and behavioral measures that can be applied to assess mental workload in pilots, and to discuss their advantages and disadvantages for this purpose. A targeted literature search was conducted to this end. The comparisons drawn in this paper reveal that a multi-method approach is preferable to relying on a single measure. In this regard, however, there is no one-size-fits-all solution and it is strongly advised to consider the selection of appropriate measures carefully for each specific research question and application context.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Wickens, C.D., Dehais, F.: Expertise in aviation. In: Ward, P., Maarten Schraagen, J., Gore, J., Roth, E.M. (eds.) The Oxford Handbook of Expertise. Oxford University Press, Oxford (2019)

    Google Scholar 

  2. Bainbridge, L.: Ironies of automation. In: Johannsen, G., Rijnsdorp, J.E. (eds.) Analysis, Design and Evaluation of Man–Machine Systems, Pergamon, pp. 129–135 (1983)

    Google Scholar 

  3. Billings, C.E.: Human-centered aviation automation: principles and guidelines. In. Ames Research Center, Moffett Field, California (1996)

    Google Scholar 

  4. Parasuraman, R.M., Mouloua, M., Hilburn, B.: adaptive aiding and adaptive task allocation enhance human-machine interaction. In: Paper Presented at the Automation Technology and Human Performance: Current Research and Trends, Norfolk, VA (1999)

    Google Scholar 

  5. Martins, A.P.G.: A review of important cognitive concepts in aviation. Aviation 20(2), 65–84 (2016). https://doi.org/10.3846/16487788.2016.1196559

    Article  Google Scholar 

  6. Charles, R.L., Nixon, J.: Measuring mental workload using physiological measures: a systematic review. Appl. Ergon. 74, 221–232 (2019). https://doi.org/10.1016/j.apergo.2018.08.028

    Article  Google Scholar 

  7. Edwards, T.: Human performance in air traffic control. University of Nottingham (2013)

    Google Scholar 

  8. Parasuraman, R., Sheridan, T.B., Wickens, C.D.: Situation awareness, mental workload, and trust in automation: viable, empirically supported cognitive engineering constructs. J. Cogn. Eng. Decis. Mak. 2(2), 140–160 (2008). https://doi.org/10.1518/155534308x284417

    Article  Google Scholar 

  9. Dekker, S., Hollnagel, E.: Human factors and folk models. Cogn. Technol. Work 6(2), 79–86 (2004). https://doi.org/10.1007/s10111-003-0136-9

    Article  Google Scholar 

  10. O'Donnell, R.D., Eggemeier, F.T.: Workload assessment methodology. In: Boff, K.R., Kaufman, L., Thomas, J.P. (eds.) Handbook of Perception and Human Performance. John Wiley & Sons, New York (1986)

    Google Scholar 

  11. Stokes, A., Kite, K.: Flight stress: stress, fatigue, and performance in aviation. Repr ed. Avebury, Aldershot (1997)

    Google Scholar 

  12. Fürstenau, N., Papenfuss, A.: Model based analysis of subjective mental workload during multiple remote tower human-in-the-loop simulations. In: Fürstenau, N. (ed.) Virtual and Remote Control Tower: Research, Design, Development, Validation, and Implementation, pp. 293–342. Springer, Cham (2022)

    Chapter  Google Scholar 

  13. Liu, H., Allen, J., Zheng, D., Chen, F.: Recent development of respiratory rate measurement technologies. Physiol. Meas. 40(7), 1–27 (2019). https://doi.org/10.1088/1361-6579/ab299e

    Article  Google Scholar 

  14. Roscoe, A.H.: Assessing pilot workload: Why measure heart rate, HRV and respiration. Biol. Psychol. 34(2), 259–287 (1992). https://doi.org/10.1016/0301-0511(92)90018-P

    Article  Google Scholar 

  15. AL-Khalidi, F.Q., Saatchi, R., Burke, D., Elphick, H., Tan, S.: Respiration rate monitoring methods: a review. Pediat. Pulmonol. 46(6), 523–529 (2011). https://doi.org/10.1002/ppul.21416

  16. Brookings, J.B., Wilson, G.F., Swain, C.R.: Psychophysiological responses to changes in workload during simulated air traffic control. Biol. Psychol. 42(3), 361–377 (1996). https://doi.org/10.1016/0301-0511(95)05167-8

    Article  Google Scholar 

  17. Mehler, B., Reimer, B., Coughlin, J.F., Dusek, J.A.: Impact of incremental increases in cognitive workload on physiological arousal and performance in young adult drivers. Transp. Res. Rec. 2138(1), 6–12 (2009). https://doi.org/10.3141/2138-02

    Article  Google Scholar 

  18. Fairclough, S.H., Venables, L., Tattersall, A.: The influence of task demand and learning on the psychophysiological response. Int. J. Psychophysiol. 56(2), 171–184 (2005). https://doi.org/10.1016/j.ijpsycho.2004.11.003

    Article  Google Scholar 

  19. Backs, R.W., Navidzadeh, H.T., Xu, X.: Cardiorespiratory indices of mental workload during simulated air traffic control. Proc. Hum. Fact. Ergon. Soc. Ann. Meet. 44(13), 89–92 (2000). https://doi.org/10.1177/154193120004401323

    Article  Google Scholar 

  20. Backs, R.W., Seljos, K.A.: Metabolic and cardiorespiratory measures of mental effort: the effects of level of difficulty in a working memory task. Int. J. Psychophysiol. 16(1), 57–68 (1994). https://doi.org/10.1016/0167-8760(94)90042-6

    Article  Google Scholar 

  21. Fairclough, S.H., Venables, L.: Prediction of subjective states from psychophysiology: a multivariate approach. Biol. Psychol. 71(1), 100–110 (2006). https://doi.org/10.1016/j.biopsycho.2005.03.007

    Article  Google Scholar 

  22. Hogervorst, M.A., Brouwer, A.-M., van Erp, J.B.F.: Combining and comparing EEG, peripheral physiology and eye-related measures for the assessment of mental workload. Front. Neurosci. 8 (2014). https://doi.org/10.3389/fnins.2014.00322

  23. Fournier, L.R., Wilson, G.F., Swain, C.R.: Electrophysiological, behavioral, and subjective indexes of workload when performing multiple tasks: manipulations of task difficulty and training. Int. J. Psychophysiol. 31(2), 129–145 (1999). https://doi.org/10.1016/S0167-8760(98)00049-X

    Article  Google Scholar 

  24. Berntson, G.G., Quigley, K.S., Norman, G.J., Lozano, D.L.: Cardiovascular psychophysiology. In: Handbook of Psychophysiology. Cambridge Handbooks in Psychology, 4 edn, pp. 183–216. Cambridge University Press, New York (2017)

    Google Scholar 

  25. Ayada, C., Toru, Ü., Korkut, Y.: The relationship of stress and blood pressure effectors. Hippokratia 19(2), 99–108 (2015)

    Google Scholar 

  26. Charkoudian, N., Rabbitts, J.A.: Sympathetic neural mechanisms in human cardiovascular health and disease. Mayo Clin. Proc. 84(9), 822–830 (2009). https://doi.org/10.4065/84.9.822

    Article  Google Scholar 

  27. Lundberg, U., et al.: Psychophysiological stress and EMG activity of the trapezius muscle. Int. J. Behav. Med. 1(4), 354–370 (1994). https://doi.org/10.1207/s15327558ijbm0104_5

    Article  Google Scholar 

  28. Veltman, J.A., Gaillard, A.W.K.: Physiological workload reactions to increasing levels of task difficulty. Ergonomics 41(5), 656–669 (1998). https://doi.org/10.1080/001401398186829

    Article  Google Scholar 

  29. Veltman, J.A., Gaillard, A.W.K.: Physiological indices of workload in a simulated flight task. Biol. Psychol. 42(3), 323–342 (1996). https://doi.org/10.1016/0301-0511(95)05165-1

    Article  Google Scholar 

  30. James, G.D., Gerber, L.M.: Measuring arterial blood pressure in humans: auscultatory and automatic measurement techniques for human biological field studies. Am. J. Hum. Biol. 30(1), e23063 (2018). https://doi.org/10.1002/ajhb.23063

    Article  Google Scholar 

  31. Xing, X., et al.: Blood pressure assessment with in-ear photoplethysmography. Physiol. Meas. 42(10), 105009 (2021). https://doi.org/10.1088/1361-6579/ac2a71

    Article  Google Scholar 

  32. Braithwaite, J.J.W., Jones, D.G., Rowe, M.R.: A guide for analysing electrodermal activity (EDA) & skin conductance responses (SCRs) for psychological experiments. In: Behavioural Brain Sciences Centre, p. 43. University of Birmingham, UK (2015)

    Google Scholar 

  33. Boucsein, W., et al.: Society for psychophysiological research ad hoc committee on electrodermal measures: publication recommendations for electrodermal measurements. Psychophysiology 49(8), 1017–1034 (2012). https://doi.org/10.1111/j.1469-8986.2012.01384.x

    Article  Google Scholar 

  34. Jackson, A.F., Bolger, D.J.: The neurophysiological bases of EEG and EEG measurement: a review for the rest of us. Psychophysiology 51(11), 1061–1071 (2014). https://doi.org/10.1111/psyp.12283

    Article  Google Scholar 

  35. Radüntz, T.: Signal quality evaluation of emerging EEG devices. Front. Physiol. 9, 98 (2018). https://doi.org/10.3389/fphys.2018.00098

    Article  Google Scholar 

  36. Wascher, E., et al.: Neuroergonomics on the go: An evaluation of the potential of mobile EEG for workplace assessment and design. Hum. Fact., 187208211007707 (2021). https://doi.org/10.1177/00187208211007707

  37. Dussault, C., Jouanin, J.-C., Guezennec, C.-Y.: EEG and ECG changes during selected flight sequences. Aviat. Space Environ. Med. 75(10), 889–897 (2004)

    Google Scholar 

  38. Hamann, A., Carstengerdes, N.: Investigating mental workload-induced changes in cortical oxygenation and frontal theta activity during simulated flights. Sci. Rep. 12(1), 6449 (2022). https://doi.org/10.1038/s41598-022-10044-y

    Article  Google Scholar 

  39. Puma, S., Matton, N., Paubel, P.-V., Raufaste, É., El-Yagoubi, R.: Using theta and alpha band power to assess cognitive workload in multitasking environments. Int. J. Psychophysiol. Off. J. Int. Organ. Psychophysiol. 123, 111–120 (2018). https://doi.org/10.1016/j.ijpsycho.2017.10.004

    Article  Google Scholar 

  40. Dehais, F., et al.: Monitoring pilot’s mental workload using ERPs and spectral power with a six-dry-electrode EEG system in real flight conditions. Sensors (Basel, Switzerland) 19(6), 1324 (2019). https://doi.org/10.3390/s19061324

    Article  Google Scholar 

  41. Holm, A., Lukander, K., Korpela, J., Sallinen, M., Müller, K.M.I.: Estimating brain load from the EEG. Sci. World J. 9, 639–651 (2009). https://doi.org/10.1100/tsw.2009.83

    Article  Google Scholar 

  42. Roy, R.N., Bonnet, S., Charbonnier, S., Campagne, A.: Mental fatigue and working memory load estimation: interaction and implications for EEG-based passive BCI. In: Conference Proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 6607–6610 (2013). https://doi.org/10.1109/embc.2013.6611070

  43. Radüntz, T., Meffert, B.: User experience of 7 mobile electroencephalography devices: comparative study. JMIR Mhealth Uhealth 7(9), e14474 (2019). https://doi.org/10.2196/14474

    Article  Google Scholar 

  44. Huppert, T.J., Franceschini, M.A., Boas, D.A.: Noninvasive imaging of cerebral activation with diffuse optical tomography. In: Frostig, R. (ed.) In Vivo Optical Imaging of Brain Function, pp. 209–238. CRC Press (2009)

    Google Scholar 

  45. Huppert, T.J.: Commentary on the statistical properties of noise and its implication on general linear models in functional near-infrared spectroscopy. Neurophotonics 3(1), 010401 (2016). https://doi.org/10.1117/1.NPh.3.1.010401

    Article  Google Scholar 

  46. Liu, T., Pelowski, M., Pang, C., Zhou, Y., Cai, J.: Near-infrared spectroscopy as a tool for driving research. Ergonomics 59(3), 368–379 (2015). https://doi.org/10.1080/00140139.2015.1076057

    Article  Google Scholar 

  47. Barker, J.W., Aarabi, A., Huppert, T.J.: Autoregressive model based algorithm for correcting motion and serially correlated errors in fNIRS. Biomed. Opt. Express 4(8), 1366–1379 (2013). https://doi.org/10.1364/boe.4.001366

    Article  Google Scholar 

  48. Causse, M., Chua, Z.K., Peysakhovich, V., Del Campo, N., Matton, N.: Mental workload and neural efficiency quantified in the prefrontal cortex using fNIRS. Sci. Rep. 7(1), 5222 (2017). https://doi.org/10.1038/s41598-017-05378-x

    Article  Google Scholar 

  49. Geissler, C.F., Schneider, J., Frings, C.: Shedding light on the prefrontal correlates of mental workload in simulated driving: a functional near-infrared spectroscopy study. Sci. Rep. 11(1), 705 (2021). https://doi.org/10.1038/s41598-020-80477-w

    Article  Google Scholar 

  50. Causse, M., Chua, Z.K., Rémy, F.: Influences of age, mental workload, and flight experience on cognitive performance and prefrontal activity in private pilots: a fNIRS study. Sci. Rep. 9(1), 7688 (2019). https://doi.org/10.1038/s41598-019-44082-w

    Article  Google Scholar 

  51. Nguyen, T., Ahn, S., Jang, H., Jun, S.C., Kim, J.G.: Utilization of a combined EEG/NIRS system to predict driver drowsiness. Sci. Rep. 7, 43933 (2017). https://doi.org/10.1038/srep43933

    Article  Google Scholar 

  52. Sammito, S., Thielmann, B., Klussmann, A., Deußen, A., Braumann, K.-M., Böckelmann, I.: S2k-Leitlinie Nutzung der Herzschlagfrequenz und der Herzfrequenzvariabilität in der Arbeitsmedizin und der Arbeitswissenschaft: AWMF-RegNr 002/042 (2021)

    Google Scholar 

  53. Kingsley, M., Lewis, M.J., Marson, R.E.: Comparison of Polar 810s and an ambulatory ECG system for RR interval measurement during progressive exercise. Int. J. Sports Med. 26(1), 39–44 (2005). https://doi.org/10.1055/s-2004-817878

    Article  Google Scholar 

  54. Nunan, D., Jakovljevic, D.G., Donovan, G., Hodges, L.D., Sandercock, G.R., Brodie, D.A.: Levels of agreement for RR intervals and short-term heart rate variability obtained from the Polar S810 and an alternative system. Eur. J. Appl. Physiol. 103(5), 529–537 (2008). https://doi.org/10.1007/s00421-008-0742-6

    Article  Google Scholar 

  55. Radespiel-Troger, M., Rauh, R., Mahlke, C., Gottschalk, T., Muck-Weymann, M.: Agreement of two different methods for measurement of heart rate variability. Clin. Auton. Res. 13(2), 99–102 (2003). https://doi.org/10.1007/s10286-003-0085-7

    Article  Google Scholar 

  56. Treiber, F.A., Musante, L., Hartdagan, S., Davis, H., Levy, M., Strong, W.B.: Validation of a heart rate monitor with children in laboratory and field settings. Med. Sci. Sports Exerc. 21(3), 338–342 (1989)

    Article  Google Scholar 

  57. Hettinger, T., Wobbe, G.: Kompendium der Arbeitswissenschaft. Kiehl Verlag, Ludwigshafen (1993)

    Google Scholar 

  58. Finsen, L., Søgaard, K., Jensen, C., Borg, V., Christensen, H.: Muscle activity and cardiovascular response during computer-mouse work with and without memory demands. Ergonomics 44(14), 1312–1329 (2001). https://doi.org/10.1080/00140130110099065

    Article  Google Scholar 

  59. Malik, M., et al.: Heart rate variability: standards of measurement, physiological interpretation, and clinical use. Eur. Heart J. 17(3), 354–381 (1996). https://doi.org/10.1093/oxfordjournals.eurheartj.a014868

    Article  Google Scholar 

  60. Nunan, D., Sandercock, G.R.H., Brodie, D.: A quantitative systematic review of normal values for short-term heart rate variability in healthy adults. Pace-Pacing Clin. Electrophysiol. 33(11), 1407–1417 (2010). https://doi.org/10.1111/j.1540-8159.2010.02841.x

    Article  Google Scholar 

  61. Schafer, A., Vagedes, J.: How accurate is pulse rate variability as an estimate of heart rate variability? a review on studies comparing photoplethysmographic technology with an electrocardiogram. Int. J. Cardiol. 166(1), 15–29 (2013). https://doi.org/10.1016/j.ijcard.2012.03.119

    Article  Google Scholar 

  62. Lohani, M., Payne, B.R., Strayer, D.L.: A review of psychophysiological measures to assess cognitive states in real-world driving. Front. Hum. Neurosci. 13, 57 (2019). https://doi.org/10.3389/fnhum.2019.00057

    Article  Google Scholar 

  63. Sammito, S., Bockelmann, I.: Analysis of heart rate variability: mathematical description and practical application. Herz 40, 76–84 (2015). https://doi.org/10.1007/s00059-014-4145-7

    Article  Google Scholar 

  64. Duchowski, A.T.: Eye Tracking Methodology: Theory and Practice, 3rd edn. Springer, Heidelberg (2007)

    Google Scholar 

  65. Goldberg, J., Wichansky, A.: Eye tracking in usability evaluation: A practitioner's guide. (2003)

    Google Scholar 

  66. Glaholt, M.G.: Eye tracking in the cockpit: a review of the relationships between eye movements and the aviator’s cognitive state. In. Defence Research and Development Canada, Toronto, Canada (2014)

    Google Scholar 

  67. Tole, J.R., Stephens, A.T., Vivaudou, M., Ephrath, A.R., Young, L.R.: Visual scanning behavior and pilot workload. In. NASA (1983)

    Google Scholar 

  68. Faulhaber, A.K., Friedrich, M.: Eye-tracking metrics as an indicator of workload in commercial single-pilot operations. In: Longo, L., Leva, M.C. (eds.) H-WORKLOAD 2019. CCIS, vol. 1107, pp. 213–225. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32423-0_14

    Chapter  Google Scholar 

  69. Di Nocera, F., Camilli, M., Terenzi, M.: A random glance at the flight deck: pilots’ scanning strategies and the real-time assessment of mental workload. J. Cogn. Eng. Dec. Mak. 1(3), 271–285 (2007). https://doi.org/10.1518/155534307X255627

    Article  Google Scholar 

  70. Stanton, N.A., Salmon, P.M., Rafferty, L.A., Walker, G.H., Baber, C., Jenkins, D.P.: Human Factors Methods: A Practical Guide for Engineering and Design, 2nd edn. CRC Press, London (2013)

    Google Scholar 

  71. Wierwille, W.W., Eggemeier, F.T.: Recommendations for mental workload measurement in a test and evaluation environment. Hum. Factors 35(2), 263–281 (1993). https://doi.org/10.1177/001872089303500205

    Article  Google Scholar 

  72. Wickens, C.D.: Multiple resources and performance prediction. Theor. Issues Ergon. Sci. 3(2), 159–177 (2002). https://doi.org/10.1080/14639220210123806

    Article  Google Scholar 

  73. Wickens, C.D.: Multiple resources and mental workload. Hum. Factors 50(3), 449–455 (2008). https://doi.org/10.1518/001872008x288394

    Article  Google Scholar 

  74. Casali, J.G., Wierwille, W.W.: A comparison of rating scale, secondary-task, physiological, and primary-task workload estimation techniques in a simulated flight task emphasizing communications load. Hum. Factors 25(6), 623–641 (1983). https://doi.org/10.1177/001872088302500602

    Article  Google Scholar 

  75. Casali, J.G., Wierwille, W.W.: On the measurement of pilot perceptual workload: a comparison of assessment techniques addressing sensitivity and intrusion issues. Ergonomics 27(10), 1033–1050 (1984). https://doi.org/10.1080/00140138408963584

    Article  Google Scholar 

  76. Wierwille, W.W., Connor, S.A.: Evaluation of 20 workload measures using a psychomotor task in a moving-base aircraft simulator. Hum. Factors 25(1), 1–16 (1983). https://doi.org/10.1177/001872088302500101

    Article  Google Scholar 

  77. Hart, S.G.: Nasa-task load index (NASA-TLX); 20 years later. Proc. Hum. Fact. Ergon. Soc. Ann. Meet. 50(9), 904–908 (2006). https://doi.org/10.1177/154193120605000909

    Article  Google Scholar 

  78. Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. In: Hancock, P.A., Meshkati, N. (eds.) Advances in Psychology, North-Holland, vol. 52. pp. 139–183 (1988)

    Google Scholar 

  79. Tattersall, A.J., Foord, P.S.: An experimental evaluation of instantaneous self-assessment as a measure of workload. Ergonomics 39(5), 740–748 (1996). https://doi.org/10.1080/00140139608964495

    Article  Google Scholar 

  80. Zimeo Morais, G.A., Balardin, J.B., Sato, J.R.: fNIRS optodes’ location decider (fOLD): a toolbox for probe arrangement guided by brain regions-of-interest. Sci Rep 8(1), 3341 (2018). https://doi.org/10.1038/s41598-018-21716-z

    Article  Google Scholar 

  81. Biella, M., Wies, M.: Human performance envelope: overview of the project and technical results. In: Paper presented at the Future Sky Safety Final Conference, FSS on Final Approach, Brüssel, Belgium (2018)

    Google Scholar 

Download references

Acknowledgements

DLR’s research work was financed with funding from the German Federal Ministry of Defence. Stefan Sammito is an active Bundeswehr Medical Service officer and works for the German Federal Ministry of Defence. This paper reflects the opinion of the authors and not necessarily the opinion of the German Federal Ministry of Defence or the Surgeon General of the German Air Force.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Hilke Boumann .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Boumann, H., Hamann, A., Biella, M., Carstengerdes, N., Sammito, S. (2023). Suitability of Physiological, Self-report and Behavioral Measures for Assessing Mental Workload in Pilots. In: Harris, D., Li, WC. (eds) Engineering Psychology and Cognitive Ergonomics. HCII 2023. Lecture Notes in Computer Science(), vol 14017. Springer, Cham. https://doi.org/10.1007/978-3-031-35392-5_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35392-5_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35391-8

  • Online ISBN: 978-3-031-35392-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics