Impact of Automation and Task Load on Unmanned System Operator’s Eye Movement Patterns

  • Cali M. Fidopiastis
  • Julie Drexler
  • Daniel Barber
  • Keryl Cosenzo
  • Michael Barnes
  • Jessie Y. C. Chen
  • Denise Nicholson
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5638)


Eye tracking under naturalistic viewing conditions may provide a means to assess operator workload in an unobtrusive manner. Specifically, we explore the use of a nearest neighbor index of workload calculated using eye fixation patterns obtained from operators navigating an unmanned ground vehicle under different task loads and levels of automation. Results showed that fixation patterns map to the operator’s experimental condition suggesting that systematic eye movements may characterize each task. Further, different methods of calculating the workload index are highly correlated, r(46) = .94, p = .01. While the eye movement workload index matches operator reports of workload based on the NASA TLX, the metric fails on some instances. Interestingly, these departure points may relate to the operator’s perceived attentional control score. We discuss these results in relation to automation triggers for unmanned systems.


Adaptive Automation Unmanned Ground Systems Eye Tracking Workload 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barnes, M.J., Cosenzo, K.A.: Human robot teams as soldier augmentation in future battlefields: An overview. In: Schmorrow, D. (ed.) Foundations of Augmented Cognition, pp. 700–709. Lawrence Erlbaum, Mawah (2005)Google Scholar
  2. 2.
    Committee on Army Unmanned Ground Vehicle Technology, National Research Council: Technology Development for Army Unmanned Ground Vehicles. The National Academies Press, Washington, D.C. (2002)Google Scholar
  3. 3.
    Kortenkamp, D., Burridge, R., Bonasso, P., Schrenkenghost, D., Hudson, M.: An intelligent software architecture for semiautonomous robot control. In: Autonomy Control Software Workshop, Autonomous Agents 1999, Seattle, WA (1999)Google Scholar
  4. 4.
    Scerbo, M.: Adaptive automation. In: Neuroergonomics: The Brain at Work, pp. 239–252. Oxford University Press, NY (2007)Google Scholar
  5. 5.
    Scerbo, M.W.: Theoretical perspectives in adaptive automation. In: Parasuraman, R., Mouloua, M. (eds.) Automation and human performance: Theory and applications, pp. 37–63. Lawrence Erlbaum Associates, Mahwah (1996)Google Scholar
  6. 6.
    Parasuraman, R., Hancock, P.A.: Adaptive workload and control. In: Hancock, P.A., Desmond, P.A. (eds.) Workload and Fatigue, pp. 305–320. Lawrence Erlbaum Associates, Mahwah (2001)Google Scholar
  7. 7.
    Kaber, D.B., Endsley, M.R.: The effects of level of automation and adaptation on human performance, situation awareness and workload in a dynamic control task. Theoretical Issues in Ergonomics Science 5(2), 113–153 (2004)CrossRefGoogle Scholar
  8. 8.
    Byrne, E.V., Parasuraman, R.: Psychophysiology and adaptive automation. Biological Psychology 42, 268–279 (1996)CrossRefGoogle Scholar
  9. 9.
    Chen, J.Y.C., Haas, E.C., Pillalamarri, K., Jacobson, C.N.: Human-robot interface: Issues in operator performance, interface design, and technologies. In: ARL Technical Report, ARL-TR-3834. US Army Research Laboratory, Aberdeen Proving Ground, MD (2006)Google Scholar
  10. 10.
    Chen, J.Y.C., Terrence, P.I.: Effects of imperfect automation on concurrent performance of Gunner’s and Robotics Operator’s tasks in a simulated mounted environment. In: ARL Technical Report ARL-TR-4455. US Army Research Laboratory, Aberdeen Proving Ground, MD (2008)Google Scholar
  11. 11.
    Parasuraman, R., Barnes, M., Cosenzo, K.: Adaptive automation for human-robot teaming in future command and control systems. The International C2 Journal 1(2), 43–68 (2007)Google Scholar
  12. 12.
    Chen, J.Y.C., Drexler, J.M., Sciarini, L.W., Cosenzo, K.A., Barnes, M.J., Nicholson, D.: Operator workload and heart-rate variability during a simulated reconnaissance mission with an unmanned ground vehicle. In: Proceedings of the 2008 Army Science Conference (in press, 2008)Google Scholar
  13. 13.
    Land, M.F.: Fixation strategies during active behavior: A brief history. In: van Gompel, R.P.G., Fischer, M.H., Murray, W.S., Hill, R.L. (eds.) Eye movements: A window on mind and brain, pp. 76–95. Elsevier, Oxford (2007)Google Scholar
  14. 14.
    Di Nocera, F., Camilli, M., Terenzi, M.: A random glance at the flight deck: Pilots’ scanning strategies and the real-time assessment of mental workload. Journal of Cognitive Engineering and Decision Making 1, 271–285 (2007)CrossRefGoogle Scholar
  15. 15.
    Barber, D., Davis, L., Nicholson, D., Finkelstein, N., Chen, J.Y.C.: The mixed initiative experimental (MIX) testbed for human robot interactions with varied levels of automation. In: Proceedings of the 2008 Army Science Conference (in press, 2008)Google Scholar
  16. 16.
    Martinez-conde, S., Mcknik, S.L., Hubel, D.H.: The role of fixational eye movements in visual perception. Nature Reviews 5, 229–240 (2004)CrossRefPubMedGoogle Scholar
  17. 17.
    Rayner, K.: Eye movements in reading and information processing: 20 years of research. Psychological Bulletin 124(3), 372–422 (1998)CrossRefPubMedGoogle Scholar
  18. 18.
    Salvucci, D.D., Goldburg, J.H.: Identifying fixations and saccades in eye-tracking protocols. In: Proceedings of the Eye Tracking Research and Application Symposium, pp. 71–78. ACM Press, New York (2000)CrossRefGoogle Scholar
  19. 19.
    Di Nocera, F., Terenzi, M., Camilli, M.: Another look at scanpath: Distance to nearest neighbor as a measure of mental workload. In: de Ward, D., Broohuis, K.A., Toffetti, A. (eds.) Developments in human factors in transportation, design, and evaluation, pp. 1–9. Shaker Publishing Maastricht, Netherlands (2006)Google Scholar
  20. 20.
    Derryberry, D., Reed, M.: Anxiety-related attentional biases and their regulation by attentional control. Journal of Abnormal Psychology 111, 225–236 (2002)CrossRefPubMedGoogle Scholar
  21. 21.
    Hart, S., Staveland, L.: Development of NASA TLX (Task Load Index): results of empirical and theoretical research. In: Hancock, P., Meshkati, N. (eds.) Human mental workload, pp. 139–183. Elsevier, Amsterdam (1988)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Cali M. Fidopiastis
    • 1
  • Julie Drexler
    • 1
  • Daniel Barber
    • 1
  • Keryl Cosenzo
    • 2
  • Michael Barnes
    • 2
  • Jessie Y. C. Chen
    • 2
  • Denise Nicholson
    • 1
  1. 1.Applied Cogntion and Training in Immersive Virtual Enviroments (ACTIVE) Laboratory Institute of Simulation and Training (IST)University of Central FloridaUSA
  2. 2.U.S. Army Research Laboratory (ARL) - Human Research & Engineering DirectorateUSA

Personalised recommendations