Skip to main content
SpringerLink
Log in
Book cover

International Symposium on Human Mental Workload: Models and Applications

H-WORKLOAD 2019: Human Mental Workload: Models and Applications pp 170–185Cite as

On the Use of Machine Learning for EEG-Based Workload Assessment: Algorithms Comparison in a Realistic Task

  • Conference paper
  • First Online:

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1107))

Abstract

The measurement of the mental workload during real tasks by means of neurophysiological signals is still challenging. The employment of Machine Learning techniques has allowed a step forward in this direction, however, most of the work has dealt with binary classification. This study proposed to examine the surveys already performed in the context of EEG-based workload classification and to test different machine learning algorithms on real multitasking activity like the Air Traffic Management. The results obtained on 35 professional Air Traffic Controllers showed that a KNN algorithm allows discriminating up to three workload levels (low, medium and high) with more than 84% of accuracy on average. Moreover, in such realistic employment it emerges how important is to opportunely choose the set of features to ward off that task-related confounds could affect the workload assessment.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Bibliography

  1. Wickens, C.D.: Multiple resources and mental workload. Hum. Factors 50(3), 449–455 (2008)

    Article  Google Scholar 

  2. Curry, R., Jex, H., Levison, W., Stassen, H.: Final report of control engineering group. In: Moray, N. (ed.) Mental Workload. NATO Conference Series, vol. 8, pp. 235–252. Springer, Boston (1979). https://doi.org/10.1007/978-1-4757-0884-4_13

    Chapter  Google Scholar 

  3. Gopher, D.: In defence of resources: on structures, energies, pools and the allocation of attention. In: Hockey, G.R.J., Gaillard, A.W.K., Coles, M.G.H. (eds.) Energetics and Human Information Processing. NATO ASI Series (Series D: Behavioural and Social Sciences), vol. 31, pp. 353–371. Springer, Dordrecht (1986). https://doi.org/10.1007/978-94-009-4448-0_25

    Chapter  Google Scholar 

  4. Kantowitz, B.H., Casper, P.A.: Human workload in aviation. In: Human Error in Aviation, pp. 123–153. Routledge, Abingdon (2017)

    Chapter  Google Scholar 

  5. Bargiotas, I., Nicolaï, A., Vidal, P.-P., Labourdette, C., Vayatis, N., Buffat, S.: The complementary role of activity context in the mental workload evaluation of helicopter pilots: a multi-tasking learning approach. In: Longo, L., Leva, M.C. (eds.) H-WORKLOAD 2018. CCIS, vol. 1012, pp. 222–238. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-14273-5_13

    Chapter  Google Scholar 

  6. Gerjets, P., Walter, C., Rosenstiel, W., Bogdan, M., Zander, T.O.: Cognitive state monitoring and the design of adaptive instruction in digital environments: lessons learned from cognitive workload assessment using a passive brain-computer interface approach. Front. Neurosci. 8(DEC), 1–21 (2014)

    Google Scholar 

  7. Byrne, A.: The effect of education and training on mental workload in medical education. In: Longo, L., Leva, M.C. (eds.) H-WORKLOAD 2018. CCIS, vol. 1012, pp. 258–266. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-14273-5_15

    Chapter  Google Scholar 

  8. Byrne, A.: Measurement of mental workload in clinical medicine: a review study. Anesthesiol. Pain Med. 1(2), 90 (2011)

    Article  Google Scholar 

  9. Jimenez-Molina, A., Retamal, C., Lira, H.: Using psychophysiological sensors to assess mental workload during web browsing. Sensors (Switzerland) 18(2), 1–26 (2018)

    Article  Google Scholar 

  10. Aricò, P., Borghini, G., Di Flumeri, G., Colosimo, A., Pozzi, S., Babiloni, F.: A passive brain–computer interface application for the mental workload assessment on professional air traffic controllers during realistic air traffic control tasks. Prog. Brain Res. 228, 295–328 (2016)

    Article  Google Scholar 

  11. Aricò, P., et al.: Adaptive automation triggered by EEG-based mental workload index: a passive brain-computer interface application in realistic air traffic control environment. Front. Hum. Neurosci. 10, 539 (2016)

    Article  Google Scholar 

  12. Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. Advances in psychology 52, 139–183 (1988)

    Article  Google Scholar 

  13. Reid, G.B., Nygren, T.E.: The subjective workload assessment technique: a scaling procedure for measuring mental workload. Advances in psychology 52, 185–218 (1988)

    Article  Google Scholar 

  14. Wilson, G.F.: Operator functional state assessment for adaptive automation implementation. In: Biomonitoring for Physiological and Cognitive Performance during Military Operations, vol. 5797, pp. 100–105 (2005)

    Google Scholar 

  15. Colle, H.A., Reid, G.B.: Double trade-off curves with different cognitive processing combinations: testing the cancellation axiom of mental workload measurement theory. Hum. Factors 41(1), 35–50 (1999)

    Article  Google Scholar 

  16. Borghini, G., Astolfi, L., Vecchiato, G., Mattia, D., Babiloni, F.: Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci. Biobehav. Rev. 44, 58–75 (2014)

    Article  Google Scholar 

  17. Aricò, P., Borghini, G., Di Flumeri, G., Sciaraffa, N., Colosimo, A., Babiloni, F.: Passive BCI in operational environments: insights, recent advances, and future trends. IEEE Trans. Biomed. Eng. 64(7), 1431–1436 (2017)

    Article  Google Scholar 

  18. Zhang, H., Zhu, Y., Maniyeri, J., Guan, C.: Detection of variations in cognitive workload using multi-modality physiological sensors and a large margin unbiased regression machine. In: 2014 36th Annual International Conference of the IEEE Engineering Medicine and Biology Society EMBC 2014, pp. 2985–2988 (2014)

    Google Scholar 

  19. Aricò, P., et al.: Towards a multimodal bioelectrical framework for the online mental workload evaluation. In: 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 3001–3004 (2014)

    Google Scholar 

  20. Di Flumeri, G., Aricò, P., Borghini, G., Sciaraffa, N., Di Florio, A., Babiloni, F.: The dry revolution: evaluation of three different eeg dry electrode types in terms of signal spectral features, mental states classification and usability. Sensors 19(6), 1365 (2019)

    Article  Google Scholar 

  21. Heard, J., Harriott, C.E., Adams, J.A.: A survey of workload assessment algorithms. IEEE Trans. Hum. Mach. Syst. 48(5), 434–451 (2018)

    Article  Google Scholar 

  22. Dijksterhuis, C., De Waard, D., Brookhuis, K.A., Mulder, B.L.J.M., De Jong, R., Kerick, S.E.: Classifying visuomotor workload in a driving simulator using subject specific spatial brain patterns. Front. Neurosci. 7(August), 1–11 (2013)

    Google Scholar 

  23. Bruggen, A.: An empirical investigation of the relationship between workload and performance. Manag. Decis. 53(10), 2377–2389 (2015)

    Article  Google Scholar 

  24. Mathan, S., Smart, A., Ververs, T., Feuerstein, M.: Towards an index of cognitive efficacy: EEG-based estimation of cognitive load among individuals experiencing cancerrelated cognitive decline. In: 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2010, pp. 6595–6598 (2010)

    Google Scholar 

  25. Baldwin, C.L., Penaranda, B.N.: Adaptive training using an artificial neural network and EEG metrics for within- and cross-task workload classification. Neuroimage 59(1), 48–56 (2012)

    Article  Google Scholar 

  26. Wang, Y.-T., et al.: Developing an EEG-based on-line closed-loop lapse detection and mitigation system. Front. Neurosci. 8, 321 (2014)

    Google Scholar 

  27. De Massari, D., et al.: Fast mental states decoding in mixed reality. Front. Behav. Neurosci. 8(November), 1–9 (2014)

    Google Scholar 

  28. Schultze-Kraft, M., Dähne, S., Gugler, M., Curio, G., Blankertz, B.: Unsupervised classification of operator workload from brain signals. J. Neural Eng. 13(3), 36008 (2016)

    Article  Google Scholar 

  29. Dimitrakopoulos, G.N., et al.: Task-independent mental workload classification based upon common multiband eeg cortical connectivity. IEEE Trans. Neural Syst. Rehabil. Eng. 25(11), 1940–1949 (2017)

    Article  Google Scholar 

  30. Zarjam, P., Epps, J., Chen, F., Lovell, N.H.: Estimating cognitive workload using wavelet entropy-based features during an arithmetic task. Comput. Biol. Med. 43(12), 2186–2195 (2013)

    Article  Google Scholar 

  31. Aghajani, H., Garbey, M., Omurtag, A.: Measuring mental workload with EEG+fNIRS. Front. Hum. Neurosci. 11(July), 1–20 (2017)

    Google Scholar 

  32. Rebsamen, B., Kwok, K., Penney, T.B.: Evaluation of cognitive workload from EEG during a mental arithmetic task. Proc. Hum. Factors Ergon. Soc. 5, 1342–1345 (2011)

    Article  Google Scholar 

  33. Zhang, P., Wang, X., Zhang, W., Chen, J.: Learning spatial-spectral-temporal EEG features with recurrent 3D convolutional neural networks for cross-task mental workload assessment. IEEE Trans. Neural Syst. Rehabil. Eng. 27(1), 31–42 (2019)

    Article  Google Scholar 

  34. Aricò, P., Aloise, F., Schettini, F., Salinari, S., Mattia, D., Cincotti, F.: Influence of P300 latency jitter on event related potential-based brain–computer interface performance. J. Neural Eng. 11(3), 35008 (2014)

    Article  Google Scholar 

  35. Radüntz, T.: Dual frequency head maps: a new method for indexing mental workload continuously during execution of cognitive tasks. Front. Physiol. 8(DEC), 1–15 (2017)

    Google Scholar 

  36. Jao, P.K., Chavarriaga, R., Millan, J.D.R.: Using robust principal component analysis to reduce EEG intra-trial variability. In: Proceedings Annual International Conference of the IEEE Engineering Medicine and Biology Society EMBS, vol. 2018-July, no. 1, pp. 1956–1959 (2018)

    Google Scholar 

  37. Dehais, F., et al.: Monitoring pilot’s mental workload using ERPs and spectral power with a six-dry-electrode EEG system in real flight conditions. Sensors (Switzerland) 19(6), 1324 (2019)

    Article  Google Scholar 

  38. Casson, A.J.: Artificial Neural Network classification of operator workload with an assessment of time variation and noise-enhancement to increase performance, vol. 8, no. December, pp. 1–10 (2014)

    Google Scholar 

  39. Fan, J., et al.: A step towards EEG-based brain computer interface for autism intervention. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 3767–3770 (2015)

    Google Scholar 

  40. Bellman, R., Kalaba, R.: Dynamic programming and statistical communication theory. Proc. Natl. Acad. Sci. U. S. A. 43(8), 749 (1957)

    Article  MathSciNet  MATH  Google Scholar 

  41. Ke, Y., et al.: An EEG-based mental workload estimator trained on working memory task can work well under simulated multi-attribute task. Front. Hum. Neurosci. 8(September), 1–10 (2014)

    Google Scholar 

  42. Mühl, C., Jeunet, C., Lotte, F., Hogervorst, M.A.: EEG-based workload estimation across affective contexts. Front. Neurosci. 8(June), 1–15 (2014)

    Google Scholar 

  43. Arvaneh, M., Umilta, A., Robertson, I.H.: Filter bank common spatial patterns in mental workload estimation. In: Proceedings Annual International Conference of the IEEE Engineering Medicine and Biology Society EMBS, 2015 November, pp. 4749–4752 (2015)

    Google Scholar 

  44. Yin, Z., Zhang, J.: Identification of temporal variations in mental workload using locally-linear-embedding-based EEG feature reduction and support-vector-machine-based clustering and classification techniques. Comput. Methods Programs Biomed. 115(3), 119–134 (2014)

    Article  Google Scholar 

  45. Christensen, J.C., Estepp, J.R., Wilson, G.F., Russell, C.A.: The effects of day-to-day variability of physiological data on operator functional state classification. Neuroimage 59(1), 57–63 (2012)

    Article  Google Scholar 

  46. Yang, S., Yin, Z., Wang, Y., Zhang, W., Wang, Y., Zhang, J.: Assessing cognitive mental workload via EEG signals and an ensemble deep learning classifier based on denoising autoencoders. Comput. Biol. Med. 109(April), 159–170 (2019)

    Article  Google Scholar 

  47. Radüntz, T., Fürstenau, N., Tews, A., Rabe, L., Meffert, B.: The effect of an exceptional event on the subjectively experienced workload of air traffic controllers. In: Longo, L., Leva, M.C. (eds.) H-WORKLOAD 2018. CCIS, vol. 1012, pp. 239–257. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-14273-5_14

    Chapter  Google Scholar 

  48. Edwards, T., Martin, L., Bienert, N., Mercer, J.: The relationship between workload and performance in air traffic control: exploring the influence of levels of automation and variation in task demand. In: Longo, L., Leva, M.C. (eds.) H-WORKLOAD 2017. CCIS, vol. 726, pp. 120–139. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-61061-0_8

    Chapter  Google Scholar 

  49. Arico, P., et al.: Human factors and neurophysiological metrics in air traffic control: a critical review. IEEE Rev. Biomed. Eng. 10, 250–263 (2017)

    Article  Google Scholar 

  50. Di Flumeri, G., Aricò, P., Borghini, G., Colosimo, A., Babiloni, F.: A new regression-based method for the eye blinks artifacts correction in the EEG signal, without using any EOG channel, vol. 2016 (2016)

    Google Scholar 

  51. Delorme, A., Makeig, S.: EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134(1), 9–21 (2004)

    Article  Google Scholar 

  52. Lim, W.L., Sourina, O., Wang, L.P.: STEW: simultaneous task EEG workload data set. IEEE Trans. Neural Syst. Rehabil. Eng. 26(11), 2106–2114 (2018)

    Article  Google Scholar 

  53. Nilsson, N.J., Nilsson, N.J.: Artificial Intelligence: A New Synthesis. Morgan Kaufmann, Burlington (1998)

    MATH  Google Scholar 

  54. Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12(Oct), 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  55. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  MATH  Google Scholar 

  56. Novak, D., Mihelj, M., Munih, M.: A survey of methods for data fusion and system adaptation using autonomic nervous system responses in physiological computing. Interact. Comput. 24(3), 154–172 (2012)

    Article  Google Scholar 

  57. Hernández, L.G., Mozos, O.M., Ferrández, J.M., Antelis, J.M.: EEG-based detection of braking intention under different car driving conditions. Front. Neuroinformatics 12(May), 1–14 (2018)

    Google Scholar 

  58. Næs, T., Mevik, B.: Understanding the collinearity problem in regression and discriminant analysis. J. Chemom. J. Chemom. Soc. 15(4), 413–426 (2001)

    Google Scholar 

  59. Deiber, M.-P., Sallard, E., Ludwig, C., Ghezzi, C., Barral, J., Ibañez, V.: EEG alpha activity reflects motor preparation rather than the mode of action selection. Front. Integr. Neurosci. 6, 59 (2012)

    Article  Google Scholar 

  60. Wolpert, D.H., Macready, W.G.: No free lunch theorems for search. Technical report SFI-TR-95-02-010, Santa Fe Institute (1995)

    Google Scholar 

  61. Wolpaw, J.R., et al.: Brain-computer interface technology: a review of the first international meeting. IEEE Trans. Rehabil. Eng. 8(2), 164–173 (2000)

    Article  Google Scholar 

  62. Di Flumeri, G., et al.: On the use of cognitive neurometric indexes in aeronautic and air traffic management environments. In: Blankertz, B., Jacucci, G., Gamberini, L., Spagnolli, A., Freeman, J. (eds) Symbiotic Interaction. Symbiotic 2015. LNCS, vol. 9359. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24917-9_5

    Chapter  Google Scholar 

  63. Aricò, P., et al.: Human-machine interaction assessment by neurophysiological measures: a study on professional air traffic controllers. In: 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (2018)

    Google Scholar 

Download references

Acknowledgment

This work is co-financed by the European Commission by Horizon2020 projects “WORKINGAGE: Smart Working environments for all Ages” (GA n. 826232); “SIMUSAFE”: Simulator Of Behavioural Aspects For Safer Transport (GA n. 723386); “SAFEMODE” (GA n. 814961); “BRAINSAFEDRIVE: A Technology to detect Mental States during Drive for improving the Safety of the road” (Italy-Sweden collaboration) with a grant of Ministero dell’Istruzione dell’Università e della Ricerca della Repubblica Italiana.

Author information

Authors and Affiliations

  1. Department of Anatomical, Histological, Forensic and Orthopedic Sciences, Sapienza University of Rome, Rome, Italy

    Nicolina Sciaraffa

  2. BrainSigns srl, Rome, Italy

    Nicolina Sciaraffa, Pietro Aricò, Gianluca Borghini, Gianluca Di Flumeri, Antonio Di Florio & Fabio Babiloni

  3. IRCCS Fondazione Santa Lucia, Rome, Italy

    Nicolina Sciaraffa, Pietro Aricò, Gianluca Borghini & Gianluca Di Flumeri

  4. Department of Molecular Medicine, Sapienza University of Rome, Rome, Italy

    Pietro Aricò, Gianluca Borghini, Gianluca Di Flumeri & Fabio Babiloni

Authors
  1. Nicolina Sciaraffa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pietro Aricò
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gianluca Borghini
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gianluca Di Flumeri
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Antonio Di Florio
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Fabio Babiloni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicolina Sciaraffa .

Editor information

Editors and Affiliations

  1. Technological University Dublin, Dublin, Ireland

    Luca Longo

  2. Technological University Dublin, Dublin, Ireland

    Maria Chiara Leva

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sciaraffa, N., Aricò, P., Borghini, G., Flumeri, G.D., Florio, A.D., Babiloni, F. (2019). On the Use of Machine Learning for EEG-Based Workload Assessment: Algorithms Comparison in a Realistic Task. In: Longo, L., Leva, M. (eds) Human Mental Workload: Models and Applications. H-WORKLOAD 2019. Communications in Computer and Information Science, vol 1107. Springer, Cham. https://doi.org/10.1007/978-3-030-32423-0_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-32423-0_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-32422-3

  • Online ISBN: 978-3-030-32423-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation