Displays for Effective Human-Agent Teaming: Evaluating Attention Management with Computational Models

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9179)


In information-dense work domains, the effectiveness of display formats in drawing attention to task-relevant information is critical. In this paper, we demonstrate a method to evaluate this capability for on-screen indicators used to proactively monitor multiple automated agents. To estimate the effectiveness of indicator formats in drawing attention to emerging problems, we compared the visual salience of indicators, as measured by computational models, to task-relevant attributes needed during proactive monitoring. The results revealed that standard formats generally do not draw attention to the information needed to identify emerging problems in multi-indicator displays, and validated the success of formats designed to more closely map task-relevant information to visual salience. We additionally report an extended saliency-based monitoring model to predict task performance from saliency and discuss implications for broader design and application.


Information visualization Intelligent and agent systems Evaluation methods and techniques 



This work was sponsored by the Office of Naval Research, Human & Bioengineered Systems (ONR 341), program officers Dr. Julie L. Marble and Dr. Jeffrey G. Morrison under contract N00014-12-C-0244. The authors thank Mr. Dan Manes and Mrs. Heather Kobus of Pacific Science & Engineering for technical assistance, and Dr. Harvey Smallman for helpful comments. The views expressed are those of the authors and do not reflect the official policy or position of the Office of Naval Research, Department of Defense, or the US Government.


  1. 1.
    Wickens, C.D., Hollands, J.G.: Engineering Psychology and Human Performance. Prentice Hall, New Jersey (2000)Google Scholar
  2. 2.
    St. John, M.F., Smallman, H.S.: Staying up to speed: four design principles for maintaining and recovering situation awareness. J. Cogn. Eng. Decis. Making 2, 118–139 (2008)CrossRefGoogle Scholar
  3. 3.
    Cook, M.B., Rieth, C.A., Ngo, M.K.: Displays for effective human-agent teaming: the role of information availability and attention management. In: Shumaker, R., Lackey, S. (eds.) VAMR 2015. LNCS, vol. 9179. Springer, Heidelberg (2015)Google Scholar
  4. 4.
    Treisman, A., Gelade, G.: A feature-integration theory of attention. Cogn. Psychol. 12, 97–136 (1980)CrossRefGoogle Scholar
  5. 5.
    Eckstein, M.P.: Visual search: a retrospective. J. Vis. 11, 14 (2011)CrossRefGoogle Scholar
  6. 6.
    Duncan, J., Humphreys, G.W.: Visual search and stimulus similarity. Psychol. Rev. 96, 433–458 (1989)CrossRefGoogle Scholar
  7. 7.
    Wolfe, J.M.: The rules of guidance in visual search. In: Kundu, M.K., Mitra, S., Mazumdar, D., Pal, S.K. (eds.) PerMIn 2012. LNCS, vol. 7143, pp. 1–10. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  8. 8.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254–1259 (1998)CrossRefGoogle Scholar
  9. 9.
    Erdem, E., Erdem, A.: Visual saliency estimation by nonlinearly integrating features using region covariances. J. Vis. 13, 1–20 (2013)CrossRefGoogle Scholar
  10. 10.
    Zhang, L., Tong, M.H., Marks, T.K., Shan, H., Cottrell, G.W.: SUN: a bayesian framework for saliency using natural statistics. J. Vis. 8, 1–20 (2008)Google Scholar
  11. 11.
    Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. Adv. Neural Inf. Process. Syst. 545–552 (2006)Google Scholar
  12. 12.
    Judd, T., Durand, F., Torralba, A.: A Benchmark of Computational Models of Saliency to Predict Human Fixations (2012).
  13. 13.
    Cook, M.B., Smallman, H.S., Rieth, C.A.: Increasing the effective span of control: advanced graphics for proactive, trend-based monitoring. IIE Trans. Occup. Ergon. Hum. Factors 2, 137–151 (2014)CrossRefGoogle Scholar
  14. 14.
    Burns, C.M.: Towards proactive monitoring in the petrochemical industry. Saf. Sci. 44, 27–36 (2006)CrossRefGoogle Scholar
  15. 15.
    Smallman, H.S., Cook, M.B.: Proactive supervisory decision support from trend-based monitoring of autonomous and automated systems: a tale of two domains. In: Shumaker, R. (ed.) VAMR 2013, Part II. LNCS, vol. 8022, pp. 320–329. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  16. 16.
    Mumaw, R.J., Roth, E.M., Vicente, K.J., Burns, C.M.: There is more to monitoring a nuclear power plan than meets the eye. Hum. Factors 42, 36–55 (2000)CrossRefGoogle Scholar
  17. 17.
    Few, S.: Information Dashboard Design. O’Reilly, Sebastopol (2006)Google Scholar
  18. 18.
    Guerlain, S., Jamieson, G.A., Bullemer, P., Blair, R.: The MPC elucidator: a case study in the design for human-automation interaction. IEEE Trans. Syst. Man Cybern. A Syst. Hum. 32, 25–40 (2004)CrossRefGoogle Scholar
  19. 19.
    Wolfe, J., Horowitz, T.: What attributes guide the deployment of visual attention and how do they do it? Nat. Rev. Neurosci. 5, 495–501 (2004)CrossRefGoogle Scholar
  20. 20.
    Bennett, K.B., Flach, J.M.: Graphical displays: implications for divided attention, focused attention, and problem solving. Hum. 34, 513–533 (1992)Google Scholar
  21. 21.
    Braseth, A.O., Veland, Ø., Welch, R.: Information rich display design. In: 4th American Nuclear Society International Topical Meeting on NPIC&HMIT (2004)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Pacific Science and Engineering GroupSan DiegoUSA

Personalised recommendations