Automatically Recognizing Intended Messages in Grouped Bar Charts

  • Richard Burns
  • Sandra Carberry
  • Stephanie Elzer
  • Daniel Chester
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7352)


Information graphics (bar charts, line graphs, grouped bar charts, etc.) often appear in popular media such as newspapers and magazines. In most cases, the information graphic is intended to convey a high-level message; this message plays a role in understanding the document but is seldom repeated in the document’s text. This paper presents our methodology for recognizing the intended message of a grouped bar chart. We discuss the types of messages communicated in grouped bar charts, the communicative signals that serve as evidence for the message, and the design and evaluation of our implemented system.


Bayesian Network Line Graph Communicative Signal Information Graphic Software Piracy 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anderson, J.R., Lebiere, C.: The Atomic Components of Thought. Lawrence Erlbaum Associates, Mahwah (1998)Google Scholar
  2. 2.
    Anderson, J.R., Matessa, M., Lebiere, C.: Act-r: A theory of higher level cognition and its relation to visual attenion. Human-Computer Interaction 12, 439–462 (1997)CrossRefGoogle Scholar
  3. 3.
    Burns, R., Elzer, S., Carberry, S.: Modeling relative task effort for grouped bar charts. In: Taatgen, N., van Rijn, H. (eds.) Proceedings of the 31st Annual Conference of the Cognitive Science Society, pp. 2292–2297. Cognitive Science Society, Austin (2009)Google Scholar
  4. 4.
    Carberry, S., Elzer, S., Demir, S.: Information graphics: An untapped resource of digital libraries. In: Proceedings of 9th International ACM SigIR Conference on Research and Development on Information Retrieval, pp. 581–588. ACM, New York (2006)Google Scholar
  5. 5.
    Chester, D., Elzer, S.: Getting Computers to See Information Graphics So Users Do Not Have to. In: Hacid, M.-S., Murray, N.V., Raś, Z.W., Tsumoto, S. (eds.) ISMIS 2005. LNCS (LNAI), vol. 3488, pp. 660–668. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  6. 6.
    Clark, H.: Using Language. Cambridge University Press (1996)Google Scholar
  7. 7.
    Elzer, S., Carberry, S., Chester, D., Demir, S., Green, N., Zukerman, I., Trnka, K.: Exploring and exploiting the limited utility of captions in recognizing intention in information graphics. In: Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pp. 223–230 (2005)Google Scholar
  8. 8.
    Elzer, S., Carberry, S., Demir, S.: Communicative signals as the key to automated understanding of bar charts. In: Proceedings of the International Conference on the Theory and Application of Diagrams (2006)Google Scholar
  9. 9.
    Elzer, S., Carberry, S., Zukerman, I., Chester, D., Green, N., Demir, S.: A probabilistic framework for recognizing intention in information graphics. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 223–230. Association for Computational Linguistics, Morristown (2005)Google Scholar
  10. 10.
    Elzer, S., Green, N., Carberry, S., Hoffman, J.: A model of perceptual task effort for bar charts and its role in recognizing intention. International Journal on User Modeling and User-Adapted Interaction 16, 1–30 (2006)CrossRefGoogle Scholar
  11. 11.
    Fasciano, M., Lapalme, G.: Intentions in the coordinated generation of graphics and text from tabular data. Knowledge and Information Systems 2(3) (August 2000)Google Scholar
  12. 12.
    Green, N.L., Carenini, G., Kerpedjiev, S., Mattis, J., Moore, J.D., Roth, S.F.: Autobrief: an experimental system for the automatic generation of briefings in integrated text and information graphics. International Journal of Human-Computer Studies 61(1), 32–70 (2004)CrossRefGoogle Scholar
  13. 13.
    Huang, W., Tan, C.L.: A system for understanding imaged infographics and its applications. In: Proceedings of the 2007 ACM Symposium on Document Engineering, DocEng 2007, pp. 9–18. ACM, New York (2007)Google Scholar
  14. 14.
    Kerpedjiev, S., Green, N., Moore, J., Roth, S.: Saying it in graphics: from intentions to visualizations. In: Proceedings of the Symposium on Information Visualization (InfoVis 1998). IEEE Computer Society Technical Committee on Computer Graphics, pp. 97–101. IEEE (1998)Google Scholar
  15. 15.
    Larkin, J.H., Simon, H.A.: Why a diagram is (sometimes) worth a thousand words. Cognitive Science 11, 65–99 (1987)CrossRefGoogle Scholar
  16. 16.
    Mittal, V.O., Carenini, G., Moore, J.D., Roth, S.: Describing complex charts in natural language: A caption generation system. Computational Linguistics 24(3), 431–467 (1998)Google Scholar
  17. 17.
    Peebles, D., Cheng, P.C.H.: Modeling the effect of task and graphical representation on response latency in a graph reading task. Human Factors 45, 28–45 (2003)CrossRefGoogle Scholar
  18. 18.
    Pinker, S.: A theory of graph comprehension. In: Artificial Intelligence and the Future of Testing, pp. 73–126. Lawrence Erlbaum Associates, Hillsdale (1990)Google Scholar
  19. 19.
    Shah, P., Mayer, R.E., Hegarty, M.: Graphs as aids to knowledge construction: Signaling techniques for guiding the process of graph comprehension. Educational Psychology 91, 690–702 (1999)CrossRefGoogle Scholar
  20. 20.
    Wickens, C.D., Carswell, C.M.: The proximity compatibility principle: Its psychological foundation and relevance to display design. Human Factors 37, 473–494 (1995)CrossRefGoogle Scholar
  21. 21.
    Wu, P., Carberry, S., Elzer, S., Chester, D.: Recognizing the Intended Message of Line Graphs. In: Goel, A.K., Jamnik, M., Narayanan, N.H. (eds.) Diagrams 2010. LNCS, vol. 6170, pp. 220–234. Springer, Heidelberg (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Richard Burns
    • 1
  • Sandra Carberry
    • 1
  • Stephanie Elzer
    • 2
  • Daniel Chester
    • 1
  1. 1.Dept of Computer ScienceUniv. of DelawareNewarkUSA
  2. 2.Dept of Computer ScienceMillersville Univ.MillersvilleUSA

Personalised recommendations