Eye Movement as Indicators of Mental Workload to Trigger Adaptive Automation

  • Tjerk de Greef
  • Harmen Lafeber
  • Herre van Oostendorp
  • Jasper Lindenberg
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5638)


This research describes an approach to objective assessment of mental workload, by analyzing differences in pupil diameter and several aspects of eye movement (fixation time, saccade distance, and saccade speed) under different levels of mental workload. In an experiment, these aspects were measured by an eye-tracking device to examine whether these are indeed indicators for mental workload. Pupil diameter and fixation time both show a general significant increase if the mental workload increases while saccade distance and saccade speed do not show any significant differences. This assessment of mental workload could be a trigger for aiding the operator of an information system, in order to meet operational requirements.


mental workload adaptive automation eye movement pupil diameter saccade fixation time 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cannon-Bowers, J.A., Salas, E.: Making decisions under stress: implications for individual and team training. American Psychological Association, Washington (1998)CrossRefGoogle Scholar
  2. 2.
    Scerbo, M.: Theoretical perspectives on adaptive automation. In: Parasuraman, R., Mouloua, M. (eds.) Automation and human performance: theory and applications, pp. 37–63. Lawrence Erlbaum Associated Publishers, Mahwah (1996)Google Scholar
  3. 3.
    Rouse, W.B.: Adaptive Aiding for Human/Computer Control. Human Factors 30, 431–443 (1988)Google Scholar
  4. 4.
    Endsley, M., Kiris, E.: The Out-of-the-Loop Performance Problem and Level of Control in Automation. Human Factors, 381–394 (1995)Google Scholar
  5. 5.
    Parasuraman, R., Mouloua, M., Molloy, R.: Effects of adaptive task allocation on monitoring of automated systems. Human Factors 38, 665–679 (1996)CrossRefPubMedGoogle Scholar
  6. 6.
    Inagaki, T.: Situation-adaptive autonomy for time-critical takeoff decisions. International Journal of Modelling and Simulation 20, 175–180 (2000)Google Scholar
  7. 7.
    Arciszewski, H.F.R., de Greef, T.E., van Delft, J.H.: Adaptive Automation in a Naval Combat Management System. IEEE Transactions on Systems, Man, and Cybernetics Part A: Systems and Humans (in press)Google Scholar
  8. 8.
    Moray, N., Inagaki, T., Itoh, M.: Adaptive Automation, Trust, and Self-Confidence in Fault Management of Time-Critical Tasks. Journal of experimental psychology, 44–57 (2000)Google Scholar
  9. 9.
    Bailey, N.R., Scerbo, M.W., Freeman, F.G., Mikulka, P.J., Scott, L.A.: Comparison of a brain-based adaptive system and a manual adaptable system for invoking automation. Human Factors 48, 693–709 (2006)CrossRefPubMedGoogle Scholar
  10. 10.
    Wilson, G.F., Russell, C.A.: Performance enhancement in an uninhabited air vehicle task using psychophysiologically determined adaptive aiding. Human Factors 49, 1005–1018 (2007)CrossRefPubMedGoogle Scholar
  11. 11.
    Byrne, E.A., Parasuraman, R.: Psychophysiology and adaptive automation. Biological Psychology 42, 249–268 (1996)CrossRefPubMedGoogle Scholar
  12. 12.
    Veltman, J.A., Gaillard, A.W.K.: Physiological workload reactions to increasing levels of task difficulty. Ergonomics 41, 656–669 (1998)CrossRefPubMedGoogle Scholar
  13. 13.
    Prinzel, L.J., Freeman, F.G., Scerbo, M.W., Mikulka, P.J., Pope, A.T.: A closed-loop system for examining psychophysiological measures for adaptive task allocation. International Journal of Aviation Psychology 10, 393–410 (2000)CrossRefPubMedGoogle Scholar
  14. 14.
    Weert, J.C.M.: Ship operator workload assessment tool. Department of mathematics and computer science. Technical University Eindhoven, Eindhoven (2006)Google Scholar
  15. 15.
    Veltman, J.A., Gaillard, A.W.D.: Pilot workload evaluated with subjective and physiological measures. In: Brookhuis, K.A., Weikert, C., Moraal, J., de Waard, D. (eds.) Proceedings of the European Chapter of the Human Factors and Ergonomics Society, Soesterberg, the Netherlands (1996)Google Scholar
  16. 16.
    Brookings, J.B., Wilson, G.F., Swain, C.R.: Psychophysiological responses to changes in workload during simulated air traffic control. Biological Psychology 42, 361–377 (1996)CrossRefPubMedGoogle Scholar
  17. 17.
    Iqbal, S.T., Zheng, X.S., Bailey, B.P.: Task-evoked pupillary response to mental workload in human-computer interaction. In: Proceedings of the ACM Conference on Human Factors in Computing Systems, Vienna, Austria, pp. 1477–1480 (2004)Google Scholar
  18. 18.
    van Orden, K.F., Limbert, W., Makeig, S., Jung, T.: Eye activity correlates of workload during a visuospatial memory task. Human Factors 43, 111–121 (2001)CrossRefPubMedGoogle Scholar
  19. 19.
    Beatty, J.: Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychological Bulletin 91, 276–292 (1982)CrossRefPubMedGoogle Scholar
  20. 20.
    Tole, J.R., Harris, R.L., Stephens, A.T., Ephrath, A.R.: Visual scanning behavior and mental workload in aircraft pilots. Aviation, Space, and Environmental Medicine 53, 54–61 (1982)PubMedGoogle Scholar
  21. 21.
    Kramer, A.F.: Physiological metrics of mental workload: A review of recent progress. In: Damos, D.L. (ed.) Multiple-task performance, Taylor & Francis, London (1991)Google Scholar
  22. 22.
    Henderson, J.M.: Visual attention and eye movement control during reading and picture viewing. In: Rayner, K. (ed.) Eye movements and visual cognition - Scene perception and reading, pp. 260–283. Springer, New York (1992)CrossRefGoogle Scholar
  23. 23.
    Stern, J.A.: The pupil of the eye: what can it tell about mental processes? Human engineering for quality and life 8, 1–2 (1997)Google Scholar
  24. 24.
    Neerincx, M.A.: Cognitive task load design: model, methods and examples. In: Hollnagel, E. (ed.) Handbook of Cognitive Task Design, pp. 283–305. Lawrence Erlbaum Associates, Mahwah (2003)CrossRefGoogle Scholar
  25. 25.
    Boer, L.C.: Workload-watch as an element of human engineering testing and Evaluation. In: Eleventh ship control systems symposium, vol. 2, Computational mechanics publications, Southampton, United Kindom (1997)Google Scholar
  26. 26.
    Rasmussen, J.: Information Processing and Human-Machine Interaction: An Approach to Cognitive Engineering. North-Holland, Amsterdam (1986)Google Scholar
  27. 27.
    Neerincx, M.A., Besouw, N.J.P.: Cognitive task load: a function of time occupied, level of information processing and task-set switches. Industrial Ergonomics, HCI, and Applied Cognitive Psychology 6, 247–254 (2001)Google Scholar
  28. 28.
    Grootjen, M., Neerincx, M.A., Veltman, J.A.: Cognitive task load in a naval ship control centre: From identification to prediction. Ergonomics 49, 1238–1264 (2006)CrossRefPubMedGoogle Scholar
  29. 29.
    Goldberg, J.H., Kotval, X.P.: Eye movement-based evaluation of the computer interface. In: Kumar, S. (ed.) Advances in occupational ergonomics and safety, pp. 529–532. ISO press, Amsterdam (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Tjerk de Greef
    • 1
    • 3
  • Harmen Lafeber
    • 2
  • Herre van Oostendorp
    • 2
  • Jasper Lindenberg
    • 3
  1. 1.Man-Machine Interaction GroupDelft University of TechnologyDelftThe Netherlands
  2. 2.Department of Information and Computing ScienceUniversity of UtrechtUtrechtThe Netherlands
  3. 3.Department of Human FactorsTNO Defence, Security and SafetySoesterbergThe Netherlands

Personalised recommendations