A Model of Perceptual Task Effort for Bar Charts and its Role in Recognizing Intention

  • Stephanie Elzer
  • Nancy Green
  • Sandra Carberry
  • James Hoffman
Original Paper


This paper presents a model of perceptual task effort for use in hypothesizing the message that a bar chart is intended to convey. It presents our rules, based on research by cognitive psychologists, for estimating perceptual task effort, and discusses the results of an eye tracking experiment that demonstrates the validity of our model. These rules comprise a model that captures the relative difficulty that a viewer would have performing one perceptual task versus another on a specific bar chart. The paper outlines the role of our model of relative perceptual task effort in recognizing the intended message of an information graphic. Potential applications of this work include using this message to provide (1) a more complete representation of the content of the document to be used for searching and indexing in digital libraries, and (2) alternative access to the information graphic for visually impaired users or users of low-bandwidth environments.


Perceptual effort Cognitive modeling Diagrams Plan recognition 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. ACT-R: ‘The ACT-R Home Page’. (retrieved on May 18th, 2004)Google Scholar
  2. Bertin J. (1983). Semiology of Graphics. The University of Wisconsin Press, Madison, WIGoogle Scholar
  3. Boff K.R., Lincoln J.E. (1988). Engineering Data Compendium: Human Perception and Performance. AAMRL, Wright-Patterson AFB, OHGoogle Scholar
  4. Carberry S. (2001). Techniques for plan recognition. User Model. User-Adap. Interact. 11(1–2):31–48zbMATHCrossRefGoogle Scholar
  5. Card S.K., Moran T.P., Newell A. (1983). The Psychology of Human-Computer Interaction. Lawrence Erlbaum Associates Inc., Hillsdale, NJGoogle Scholar
  6. Cavanaugh J.P. (1972). Relation between the intermediate memory span and the memory search rate. Psychol. Rev. 79:525–530CrossRefGoogle Scholar
  7. Charniak E., Goldman R.P. (1993). A Bayesian model of plan recognition. Artif. Intell. 64(1):53–79CrossRefGoogle Scholar
  8. Clark H. (1996). Using Language. Cambridge University Press, CambridgeGoogle Scholar
  9. Cleveland W.S. (1985). The Elements of Graphing Data. Chapman and Hall, New YorkGoogle Scholar
  10. Corio, M., Lapalme, G.: Generation of texts for information graphics. In: Proceedings of the 7th European Workshop on Natural Language Generation EWNLG’99. pp. 49–58 (1999)Google Scholar
  11. Druzdzel M.J., van der Gaag L.C. (2000). Building probabilistic networks: Where do the numbers come from?. IEEE Trans. Knowl. Data. Eng. 12:481–486CrossRefGoogle Scholar
  12. Elzer, S., Carberry, S., Zukerman, I., Chester, D., Green, N., Demir, S.: A probabilistic framework for recognizing intention in information graphics. In: Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence (IJCAI). pp. 1042–1047 (2005)Google Scholar
  13. EPIC: The brain, cognition, and action laboratory: EPIC. (retrieved on May 18th, 2004)Google Scholar
  14. Futrelle R. (1999). Summarization of diagrams in documents. In: Mani I., Maybury M. (eds) Advances in Automated Text Summarization. MIT Press, Cambridge, pp. 403–421Google Scholar
  15. Futrelle, R., Nikolakis, N.: Efficient analysis of complex diagrams using constraint-based parsing. In: Proceedings of the Third International Conference on Document Analysis and Recognition. pp.~782–790 (1995)Google Scholar
  16. Green N., Carenini G., Kerpedjiev S., Mattis J., Moore J., Roth S. (2004). Autobrief: an experimental system for the automatic generation of briefings in integrated text and information graphics. Int. J. Hum-Comp. Stud. 61(1):32–70CrossRefGoogle Scholar
  17. Grice H.P. (1969). Utterer’s meaning and intentions. Philos. Rev. 78:147–177CrossRefGoogle Scholar
  18. Hollands J.G., Spence I. (2001). The discrimination of graphical elements. Appl. Cogn. Psychol. 15:413–431CrossRefGoogle Scholar
  19. Iverson G., Gergen M. (1997). Statistics: The Conceptual Approach. Springer-Verlag, New YorkGoogle Scholar
  20. John B.E., Newell A. (1990). Toward an engineering model of stimulus response compatibility. In: Gilmore R.W., Reeve T.G. (eds) Stimulus-Response Compatibility: An Integrated Approach. North-Holland, New York, pp. 107–115Google Scholar
  21. Kerpedjiev, S., Roth, S.F.: Mapping communicative goals into conceptual tasks to generate graphics in discourse. In: Proceedings of Intelligent User Interfaces. pp. 157–164 (2000)Google Scholar
  22. Kosslyn S. (1994). Elements of Graph Design. W. H. Freeman and Company, NYGoogle Scholar
  23. Kosslyn S.M. (1983). Ghosts in the Mind’s Machine. Norton, New YorkGoogle Scholar
  24. Kosslyn S.M. (1989). Understanding charts and graphs. Appl. Cogn. Psychol. 3:185–226CrossRefGoogle Scholar
  25. Larkin J.H., Simon H.A. (1987). Why a diagram is (sometimes) worth a thousand words. Cogn. Sci. 11, 65–99CrossRefGoogle Scholar
  26. Lohse G.L. (1993). A cognitive model for understanding graphical perception. Hum-Comp. Interact. 8:353–388CrossRefGoogle Scholar
  27. Pearl J. (1988). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers, CAGoogle Scholar
  28. Peebles D., Cheng P.C.-H. (2003). Modeling the effect of task and graphical representation on response latency in a graph reading task. Hum. Factors. 45:28–46CrossRefGoogle Scholar
  29. Perrault C.R., Allen J.F. (1980). A plan-based analysis of indirect speech acts. Am. J. Comput. Ling. 6(3–4):167–182Google Scholar
  30. Pomerantz, J.R., Kubovy, M.: Theoretical approaches to perceptual organization. In: Boff, K.R., Kaufman, L., Thomas, J.P. (eds.) Handbook of Perception and Human Performance, pp.36.1–36.46. Wiley, New York (1986)Google Scholar
  31. Russo J.E. (1978). Adaptation of cognitive processes to eye movement systems. In: Senders J.W., Fisher D.F., Monty R.A. (eds) Eye Movements and Higher Psychological Functions. Lawrence Erlbaum Associates Inc., Hillsdale, NJGoogle Scholar
  32. Shah P. (2002). Graph comprehension: the role of format, content, and individual differences. In: Anderson M., Meyer M.B., Olivier P. (eds) Diagrammatic Representation and Reasoning. Springer Verlag, BerlinGoogle Scholar
  33. Shah P., Mayer R.E., Hegarty M. (1999). Graphs as aids to knowledge construction: signaling techniques for guiding the process of graph comprehension. J. Educ. Psychol. 1991(4):690–702CrossRefGoogle Scholar
  34. Simkin D., Hastie R. (1987). An information-processing analysis of graph perception. J. Am. Statist. Assoc. 82(398):454–465CrossRefGoogle Scholar
  35. Sripada, S.G., Reiter, E., Hunter, J., Yu, J.: Segmenting time series for weather forecasting. In: Macintosh, A., Ellis, R., Coenen, F. (eds.) Proceedings of ES2002. pp. 193–206 (2002)Google Scholar
  36. Tufte E.R. (1983). The Visual Display of Quantitative Information. Graphics Press, Cheshire, CTGoogle Scholar
  37. Welford A.T. (1973). Attention, strategy, and reaction time. In: Kornblum S. (eds) Attention and Performance IV. Academic, New York, pp. 37–54Google Scholar
  38. Wickens C.D., Carswell C.M. (1995). The proximity compatibility principle: its psychological foundation and relevance to display design. Hum Factors 37(3):473–494CrossRefGoogle Scholar
  39. Yu, J., Hunter, J., Reiter, E., Sripada, S.: Recognising visual patterns to communicate gas turbine time-series data. In: Macintosh, A., Ellis, R., Coenen, F. (eds.) Proceedings of ES2002. pp. 105–118 (2002)Google Scholar
  40. Zacks J., Tversky B. (1999). Bars and lines: a study of graphic communication. Mem Cogn 27(6):1073–1079Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2006

Authors and Affiliations

  • Stephanie Elzer
    • 1
  • Nancy Green
    • 2
  • Sandra Carberry
    • 3
  • James Hoffman
    • 4
  1. 1.Dept of Computer ScienceMillersville UnivMillersvilleUSA
  2. 2.Dept of Mathematical SciencesUniversity of North Carolina at GreensboroGreensboroUSA
  3. 3.Department of Computer ScienceUniversity of DelawareNewarkUSA
  4. 4.Department of PsychologyUniversity of DelawareNewarkUSA

Personalised recommendations