Displays for Effective Human-Agent Teaming: Evaluating Attention Management with Computational Models
In information-dense work domains, the effectiveness of display formats in drawing attention to task-relevant information is critical. In this paper, we demonstrate a method to evaluate this capability for on-screen indicators used to proactively monitor multiple automated agents. To estimate the effectiveness of indicator formats in drawing attention to emerging problems, we compared the visual salience of indicators, as measured by computational models, to task-relevant attributes needed during proactive monitoring. The results revealed that standard formats generally do not draw attention to the information needed to identify emerging problems in multi-indicator displays, and validated the success of formats designed to more closely map task-relevant information to visual salience. We additionally report an extended saliency-based monitoring model to predict task performance from saliency and discuss implications for broader design and application.
KeywordsInformation visualization Intelligent and agent systems Evaluation methods and techniques
This work was sponsored by the Office of Naval Research, Human & Bioengineered Systems (ONR 341), program officers Dr. Julie L. Marble and Dr. Jeffrey G. Morrison under contract N00014-12-C-0244. The authors thank Mr. Dan Manes and Mrs. Heather Kobus of Pacific Science & Engineering for technical assistance, and Dr. Harvey Smallman for helpful comments. The views expressed are those of the authors and do not reflect the official policy or position of the Office of Naval Research, Department of Defense, or the US Government.
- 1.Wickens, C.D., Hollands, J.G.: Engineering Psychology and Human Performance. Prentice Hall, New Jersey (2000)Google Scholar
- 3.Cook, M.B., Rieth, C.A., Ngo, M.K.: Displays for effective human-agent teaming: the role of information availability and attention management. In: Shumaker, R., Lackey, S. (eds.) VAMR 2015. LNCS, vol. 9179. Springer, Heidelberg (2015)Google Scholar
- 10.Zhang, L., Tong, M.H., Marks, T.K., Shan, H., Cottrell, G.W.: SUN: a bayesian framework for saliency using natural statistics. J. Vis. 8, 1–20 (2008)Google Scholar
- 11.Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. Adv. Neural Inf. Process. Syst. 545–552 (2006)Google Scholar
- 12.Judd, T., Durand, F., Torralba, A.: A Benchmark of Computational Models of Saliency to Predict Human Fixations (2012). http://saliency.mit.edu/
- 17.Few, S.: Information Dashboard Design. O’Reilly, Sebastopol (2006)Google Scholar
- 20.Bennett, K.B., Flach, J.M.: Graphical displays: implications for divided attention, focused attention, and problem solving. Hum. 34, 513–533 (1992)Google Scholar
- 21.Braseth, A.O., Veland, Ø., Welch, R.: Information rich display design. In: 4th American Nuclear Society International Topical Meeting on NPIC&HMIT (2004)Google Scholar