Communicative Signals as the Key to Automated Understanding of Simple Bar Charts

  • Stephanie Elzer
  • Sandra Carberry
  • Seniz Demir
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4045)


This paper discusses the types of communicative signals that frequently appear in simple bar charts and how we exploit them as evidence in our system for inferring the intended message of an information graphic. Through a series of examples, we demonstrate the impact that various types of communicative signals, namely salience, captions and estimated perceptual task effort, have on the intended message inferred by our implemented system.


Bayesian Network Perceptual Task Communicative Signal Information Graphic Graphic Designer 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bertin, J.: Semiology of Graphics. The University of Wisconsin Press, Madison (1983)Google Scholar
  2. 2.
    Carberry, S.: Techniques for plan recognition. User Modeling and User-Adapted Interaction 11(1–2), 31–48 (2001)MATHCrossRefGoogle Scholar
  3. 3.
    Card, S.K., Moran, T.P., Newell, A.: The Psychology of Human-Computer Interaction. Lawrence Erlbaum Associates, Inc., Hillsdale (1983)Google Scholar
  4. 4.
    Charniak, E., Goldman, R.P.: A bayesian model of plan recognition. Artificial Intelligence 64(1), 53–79 (1993)CrossRefGoogle Scholar
  5. 5.
    Chester, D., Elzer, S.: Getting computers to see information graphics so users do not have to. In: Hacid, M.-S., Murray, N.V., Raś, Z.W., Tsumoto, S. (eds.) ISMIS 2005. LNCS (LNAI), vol. 3488, pp. 660–668. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  6. 6.
    Clark, H.: Using Language. Cambridge University Press, Cambridge (1996)CrossRefGoogle Scholar
  7. 7.
    Cleveland, W.S.: The Elements of Graphing Data. Chapman and Hall, New York (1985)Google Scholar
  8. 8.
    Corio, M., Lapalme, G.: Generation of texts for information graphics. In: Proceedings of the 7th European Workshop on Natural Language Generation EWNLG 1999, pp. 49–58 (1999)Google Scholar
  9. 9.
    Druzdzel, M.J., van der Gaag, L.C.: Building probabilistic networks: ‘where do the numbers come from? IEEE Transactions on Knowledge and Data Engineering 12, 481–486 (2000)CrossRefGoogle Scholar
  10. 10.
    Elzer, S.: A Probabilistic Framework for the Recognition of Intention in Information Graphics. Ph.D thesis, University of Delaware (December 2005)Google Scholar
  11. 11.
    Elzer, S., Carberry, S., Chester, D., Demir, S., Green, N., Zukerman, I., Trnka, K.: Exploring and exploiting the limited utility of captions in recognizing intention in information graphics. In: Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL), pp. 223–230 (June 2005)Google Scholar
  12. 12.
    Elzer, S., Carberry, S., Green, N., Hoffman, J.: Incorporating perceptual task effort into the recognition of intention in information graphics. In: Blackwell, A.F., Marriott, K., Shimojima, A. (eds.) Diagrams 2004. LNCS (LNAI), vol. 2980, pp. 255–270. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  13. 13.
    Elzer, S., Carberry, S., Zukerman, I., Chester, D., Green, N., Demir, S.: A probabilistic framework for recognizing intention in information graphics. In: Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence (IJCAI), July 2005, pp. 1042–1047 (2005)Google Scholar
  14. 14.
    Green, N., Carenini, G., Kerpedjiev, S., Mattis, J., Moore, J., Roth, S.: Autobrief: an experimental system for the automatic generation of briefings in integrated text and information graphics. International Journal of Human-Computer Studies 61(1), 32–70 (2004)CrossRefGoogle Scholar
  15. 15.
    John, B.E., Newell, A.: Toward an engineering model of stimulus response compatibility. In: Gilmore, R.W., Reeve, T.G. (eds.) Stimulus-response compatibility: An integrated approach, pp. 107–115. North-Holland, New York (1990)Google Scholar
  16. 16.
    Kerpedjiev, S., Roth, S.F.: Mapping communicative goals into conceptual tasks to generate graphics in discourse. In: Proceedings of Intelligent User Interfaces, pp. 157–164 (2000)Google Scholar
  17. 17.
    Stephen, M., Kosslyn, S.M.: Understanding charts and graphs. Applied Cognitive Psychology 3, 185–226 (1989)CrossRefGoogle Scholar
  18. 18.
    Larkin, J.H., Simon, H.A.: Why a diagram is (sometimes) worth a thousand words. Cognitive Science 11, 65–99 (1987)CrossRefGoogle Scholar
  19. 19.
    Lohse, G.L.: A cognitive model for understanding graphical perception. Human-Computer Interaction 8, 353–388 (1993)CrossRefGoogle Scholar
  20. 20.
    Merriam-Webster On-Line Thesaurus,
  21. 21.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers, San Francisco (1988)Google Scholar
  22. 22.
    Peebles, D., Cheng, P.C.-H.: Modeling the effect of task and graphical representation on response latency in a graph reading task. Human Factors 45, 28–46 (2003)CrossRefGoogle Scholar
  23. 23.
    Russo, J.E.: Adaptation of cognitive processes to eye movement systems. In: Senders, J.W., Fisher, D.F., Monty, R.A. (eds.) Eye movements and higher psychological functions, Lawrence Erlbaum Associates, Inc., Hillsdale (1978)Google Scholar
  24. 24.
    Tufte, E.R.: The Visual Display of Quantitative Information. Graphics Press, Cheshire (1983)Google Scholar
  25. 25.

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Stephanie Elzer
    • 1
  • Sandra Carberry
    • 2
  • Seniz Demir
    • 2
  1. 1.Dept of Computer ScienceMillersville Univ.MillersvilleUSA
  2. 2.Dept of Computer ScienceUniv. of DelawareNewarkUSA

Personalised recommendations