Helping People with Visual Impairments Gain Access to Graphical Information Through Natural Language: The iGraph System

  • Leo Ferres
  • Avi Parush
  • Shelley Roberts
  • Gitte Lindgaard
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4061)


Much numerical information is visualized in graphs. However, this is a medium that is problematic for people with visual impairments. We have developed a system called iGraph which provides short verbal descriptions of the information usually depicted in graphs. This system was used as a preliminary solution that was validated through a process of User Needs Analysis (UNA). This process provided some basic data on the needs of people with visual impairments in terms of the components and the language to be used for graph comprehension and also validated our initial approach. The UNA provided important directions for the further development of iGraph particularly in terms of interactive querying of graphs.


Visual Impairment Line Graph Input Graph Natural Language Generation Interactive Querying 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Shah, P., Carpenter, P.A.: Conceptual limitations in comprehending line graphs. Journal of Experimental Psychology: General 124(1), 43–61 (1995)CrossRefGoogle Scholar
  2. 2.
    Bertin, J.: Semiology of Graphics. The University of Wisconsin Press, Madison (1983)Google Scholar
  3. 3.
    Carpenter, P., Shah, P.: A model of the perceptual and conceptual processes in graph comprehension. Journal of Experimental Psychology: Applied, 4, 75–100 (1998)CrossRefGoogle Scholar
  4. 4.
    Pinker, S.: A theory of graph comprehension. In: Freedle, R. (ed.) Artificial intelligence and the future of testing, pp. 73–126. Lawrence Erlbaum, Hillsdale (1990)Google Scholar
  5. 5.
    Trickett, S.B., Ratwani, R.M., Trafton, J.G.: Real-world graph comprehension: High-level questions, complex graphs, and spatial cognition (under review)Google Scholar
  6. 6.
    Kosslyn, S.: Understanding charts and graphs. Applied Cognitive Psychology 3, 185–226 (1989)CrossRefGoogle Scholar
  7. 7.
    Lohse, J.: A cognitive model for the perception and understanding of graphs. In: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 137–144. ACM Press, New York (1991)Google Scholar
  8. 8.
    Foster, M.: Evaluating models of visual comprehension. In: Proceedings of EuroCogsci 2003 (2003)Google Scholar
  9. 9.
    Flowers, J.H., Hauer, T.A.: The ear’s versus the eye’s potential to assess characteristics of numeric data: Are we too visuocentric? Behavior Research Methods, Instruments, & Computers 24(2), 258–264 (1992)CrossRefGoogle Scholar
  10. 10.
    Flowers, J.H.: Thirteen years of reflection on auditory graphing: Promises, pitfalls, and potential new directions. In: Proceedings of ICAD 2005-Eleventh Meeting of the International Conference on Auditory Display (2005)Google Scholar
  11. 11.
    Brown, L., Brewster, S., Ramloll, R., Burton, M., Riedel, B.: Design guidelines for audio presentation of graphs and tables. In: Proceedings of ICAD 2003 Workshop on Auditory Displays In Assistive Technologies, University of Boston (2003)Google Scholar
  12. 12.
    McCoy, K.F., Carberry, M.S., Roper, T., Green, N.: Towards generating textual summaries of graphs. In: Proceedings of the 1st International Conference on Universal Access in Human-Computer Interaction, ACM Press, New York (2001)Google Scholar
  13. 13.
    Fasciano, M., Lapalme, G.: Postgraphe: a system for the generation of statistical graphics and text. In: Proceedings of the 8th International Workshop on Natural Language Generation (INLG 1996) (1996)Google Scholar
  14. 14.
    Corio, M., Lapalme, G.: Generation of texts for information graphics. In: Saint-Dizier, P. (ed.) Proceedings of the 7th. European Workshop on Natural Language Generation (EWNLG 1999), Toulouse, pp. 49–58 (1999)Google Scholar
  15. 15.
    Fasciano, M., Lapalme, G.: Intentions in the coordinated generation of graphics and text from tabular data. Knowl. Inf. Syst. 2(3), 310–339 (2000)MATHCrossRefGoogle Scholar
  16. 16.
    Mittal, V.O., Carenini, G., Moore, J.D., Roth, S.: Describing complex charts in natural language: a caption generation system. Computational Linguistics 24(3), 431–467 (1998)Google Scholar
  17. 17.
    Kerpedjiev, S.M.: Automatic generation of multimodal weather reports from datasets. In: Proceedings of the third conference on Applied natural language processing, pp. 48–55. Association for Computational Linguistics (1992)Google Scholar
  18. 18.
    Goldberg, E., Driedger, N., Kittredge, R.: Using natural-language processing to produce weather forecasts. IEEE Expert 9, 45–53 (1994)CrossRefGoogle Scholar
  19. 19.
    Reiter, E., Dale, R.: Building applied natural language generation systems. Journal of Natural Language Engineering 3(1), 57–87 (1997)CrossRefGoogle Scholar
  20. 20.
    Reiter, E., Dale, R.: Building natural language generation systems. Cambridge University Press, Cambridge (2000)CrossRefGoogle Scholar
  21. 21.
    Yu, J., Reiter, E., Hunter, J., Sripada, S.: A new architecture for summarising time series data. In: Proceedings of INLG 2004 Poster Session, pp. 47–50 (2004)Google Scholar
  22. 22.
    Yu, J., Reiter, E., Hunter, J., Sripada, S.: Sumtime-turbine: A knowledge-based system to communicate time series data in the gas turbine domain. In: Proceedings of IEA/AIE-2003, pp. 379–384 (2003)Google Scholar
  23. 23.
    Sripada, S., Reiter, E., Hunter, J., Yu, J.: Summarising neonatal time series data. In: Proceedings of 2003 Conference of the European Chapter of the Association for Computation Linguistics, Companion Volume, pp. 734–743 (2003)Google Scholar
  24. 24.
    Dahlbäck, N., Jönsson, A., Ahrenberg, L.: Wizard of oz studies – why and how. Knowledge-Based Systems 6(4), 258–266 (1993)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Leo Ferres
    • 1
  • Avi Parush
    • 1
  • Shelley Roberts
    • 1
  • Gitte Lindgaard
    • 1
  1. 1.Human-Oriented Technology LaboratoryCarleton UniversityOttawaCanada

Personalised recommendations