Acoustic Rendering of Data Tables Using Earcons and Prosody for Document Accessibility

  • Dimitris Spiliotopoulos
  • Panagiota Stavropoulou
  • Georgios Kouroupetroglou
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5616)


Earlier works show that using a prosody specification that is derived from natural human spoken rendition, increases the naturalness and overall acceptance of speech synthesised complex visual structures by conveying to audio certain semantic information hidden in the visual structure. However, prosody alone, although exhibits significant improvement, cannot perform adequately in the cases of very large complex data tables browsed in a linear manner. This work reports on the use of earcons and spearcons combined with prosodically enriched aural rendition of simple and complex tables. Three spoken combinations earcons+prosody, spearcons+prosody, and prosody were evaluated in order to examine how the resulting acoustic output would improve the document-to-audio semantic correlation throughput from the visual modality. The results show that the use of non-speech sounds can further improve certain qualities, such as listening effort, a crucial parameter when vocalising any complex visual structure contained in a document.


document-to-audio data tables earcons prosody Text-to-Speech ToBI document accessibility 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Freitas, D., Kouroupetroglou, G.: Speech Technologies for Blind and Low Vision Persons. Technology and Disability 20(2), 135–156 (2008)Google Scholar
  2. 2.
    Spiliotopoulos, D., Xydas, G., Kouroupetroglou, G.: Diction Based Prosody Modeling in Table-to-Speech Synthesis. In: Matoušek, V., Mautner, P., Pavelka, T. (eds.) TSD 2005. LNCS (LNAI), vol. 3658, pp. 294–301. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  3. 3.
    Spiliotopoulos, D., Xydas, G., Kouroupetroglou, G., Argyropoulos, V.: Experimentation on Spoken Format of Tables in Auditory User Interfaces. In: Proc. 11th Int. Conf. on Human-Computer Interaction (HCII 2005), USA, July 22-27 (2005)Google Scholar
  4. 4.
    Fellbaum, K., Kouroupetroglou, G.: Principles of Electronic Speech Processing with Applications for People with Disabilities. Technology and Disability 20(2), 55–85 (2008)Google Scholar
  5. 5.
    Ramel, J.-Y., Crucianou, M., Vincent, N., Faure, C.: Detection, Extraction and Representation of Tables. In: Proc. 7th Int. Conf. Doc. Analysis and Recognition (ICDAR), pp. 374–378 (2003)Google Scholar
  6. 6.
    Silva, A.C., Jorge, A.M., Torgo, L.: Design of an end-to-end method to extract information from tables. Int. J. Document Analysis and Recognition, Special issue on detection and understanding of tables and forms for document processing applications, 8(2), 144–171 (2006)CrossRefGoogle Scholar
  7. 7.
    Hurst, M.: Towards a theory of tables. Int. J. of Doc. Analysis 8(2-3), 123–131 (2006)Google Scholar
  8. 8.
    Pontelli, E., Xiong, W., Gupta, G., Karshmer, A.: A Domain Specific Language Framework for Non-visual Browsing of Complex HTML Structures. In: Proc. ACM Conf. Assistive Technologies (ASSETS), pp. 180–187 (2000)Google Scholar
  9. 9.
    Embley, D.W., Hurst, M., Lopresti, D.P., Nagy, G.: Table-processing paradigms: a research survey. Int. J. Document Analysis 8(2-3), 66–86 (2006)CrossRefGoogle Scholar
  10. 10.
    Pontelli, E., Gillan, D.J., Gupta, G., Karshmer, A.I., Saad, E., Xiong, W.: Intelligent non-visual navigation of complex HTML structures. Universal Access in the Information Society 2(1), 56–69 (2002)CrossRefGoogle Scholar
  11. 11.
    Pontelli, E., Gillan, D., Xiong, W., Saad, E., Gupta, G., Karshmer, A.: Navigation of HTML Tables, Frames, and XML Fragments. In: Proc. ACM Conf. on Assistive Technologies (ASSETS), pp. 25–32 (2002)Google Scholar
  12. 12.
    Pitt, I., Edwards, A.: An Improved Auditory Interface for the Exploration of Lists. In: ACM Multimedia 1997, pp. 51–61 (1997)Google Scholar
  13. 13.
    Hurst, M., Douglas, S.: Layout & Language: Preliminary Experiments in Assigning Logical Structure to Table Cells. In: Proc. 4th Int. Conf. Document Analysis and Recognition (ICDAR), pp. 1043–1047 (1997)Google Scholar
  14. 14.
    Filepp, R., Challenger, J., Rosu, D.: Improving the Accessibility of Aurally Rendered HTML Tables. In: Proc. ACM Conf. on Assistive Technologies (ASSETS), pp. 9–16 (2002)Google Scholar
  15. 15.
    Lim, S., Ng, Y.: An Automated Approach for Retrieving Hierarchical Data from HTML Tables. In: Proc. 8th ACM Int. Conf. Information and Knowledge Management (CIKM), pp. 466–474 (1999)Google Scholar
  16. 16.
    Yesilada, Y., Stevens, R., Goble, C., Hussein, S.: Rendering Tables in Audio: The Interaction of Structure and Reading Styles. In: Proc. ACM Conf. Assistive Technologies (ASSETS), pp. 16–23 (2004)Google Scholar
  17. 17.
    Kottapally, K., Ngo, C., Reddy, R., Pontelli, E., Son, T.C., Gillan, D.: Towards the Creation of Accessibility Agents for Non-visual Navigation of the Web. In: Proc. of the ACM Conf. on Universal Usability, Vancouver, Canada, pp. 134–141 (2003)Google Scholar
  18. 18.
    Oogane, T., Asakawa, C.: An Interactive Method for Accessing Tables in HTML. In: Proc. Intl. ACM Conf. on Assistive Technologies, pp. 126–128 (1998)Google Scholar
  19. 19.
    Brewster, S.A., Wright, P.C., Edwards, A.D.N.: An evaluation of earcons for use in auditory human-computer interfaces. In: SIGCHI Conf. on Human Factors in Computing Systems, Amsterdam (1993)Google Scholar
  20. 20.
    Lucas, P.: An evaluation of the communicative ability of auditory icons and earcons. In: Kramer, G. (ed.) Proc. ICAD 1994, Santa Fé Institute, Santa Fé, NM. Addison-Wesley, Reading (1994)Google Scholar
  21. 21.
    Lemmens, P.M.C., Bussemakers, M.P., de Haan, A.: Effects of Auditory Icons and Earcons on Visual Categorisation: The Bigger Picture. In: Proc. 2001 Int. Conf. on Auditory Display, Espoo, Finland, July 29-August 1 (2001)Google Scholar
  22. 22.
    Brewster, S., Raty, V.-P., Kortekangas, A.: Earcons as a method of providing navigational cues in a menu hierarchy. In: Proc. Int. Conf. Human Computer Interaction. Imperial College, London (1996)Google Scholar
  23. 23.
    Brewster, S.: Navigating telephone-based interfaces with earcons. In: BCS HCI 1997, UK (1997)Google Scholar
  24. 24.
    LePlâtre, G., Brewster, S.: Designing non-speech sounds to support navigation in mobile phone menus. In: Int. Conf. on Auditory Display (ICAD 2000), Atlanta, USA (1998)Google Scholar
  25. 25.
    Watson, M.: Scalable earcons: Bridging the gap between intermittent and continuous auditory displays. In: Proc. 12th Int. Conf. on Auditory Display, UK, June 20-23 (2006)Google Scholar
  26. 26.
    Walker, B.N., Nance, A., Lindsay, J.: Spearcons: speech-based earcons improve navigation performance in auditory menus. In: Proc. 12th Int. Conf. on Auditory Display (ICAD 2006), London, UK, June 20-23 (2006)Google Scholar
  27. 27.
    Xydas, G., Kouroupetrolgou, G.: Text-to-Speech Scripting Interface for Appropriate Vocalisation of E-Texts. In: Proc. 7th European Conf. Speech Communication and Technology, pp. 2247–2250 (2001)Google Scholar
  28. 28.
    Baltazani, M., Jun, S.-A.: Focus and topic intonation in Greek. In: Proc. 14th Int. Congress of Phonetic Sciences, vol. 2, pp. 1305–1308 (1999)Google Scholar
  29. 29.
    Dainora, A.: Does intonational meaning come from tones or tunes? evidence against a compositional approach. In: Proc. Speech Prosody 2002, pp. 235–238 (2002)Google Scholar
  30. 30.
    Herman, R.: Intonation and discourse structure in English: Phonological and phonetic markers of local and global discourse structure, Ph.D Thesis (1998)Google Scholar
  31. 31.
    Nakatani, C., Hirschberg, J., Grosz, B.: Discourse Structure in Spoken Language, Studies on Speech Corpora (1995)Google Scholar
  32. 32.
    Swerts, M.: Combining statistical and phonetic analyses of spontaneous discourse segmentation. In: Proc. 12th Int. Congress of Phonetic Sciences, Stockholm, August 1995, vol. 4, pp. 208–211 (1995)Google Scholar
  33. 33.
    Blattner, M.M., Sumikawa, D.A., Greenberg, R.M.: Earcons and Icons: Their Structure and Common Design Principles. Human-Computer Interaction 4(1), 11–44 (1989)CrossRefGoogle Scholar
  34. 34.
    Van Donzel, M.: Prosodic Aspects of Information Structure in Discourse. LOT dissertations, vol. 23. Holland Academic Press, The Hague (1999)Google Scholar
  35. 35.
    Den Ouden, H., Noordman, L., Terken, J.: The prosodic realization of organizational features of texts. In: Proc. Speech Prosody 2002, pp. 543–546 (2002)Google Scholar
  36. 36.
    Brewster, S.A., Wright, P.C., Edwards, A.D.N.: Experimentally derived guidelines for the creation of earcons. In: Proc. HCI 1995, Huddersfield, UK (1995)Google Scholar
  37. 37.
    Caldwell, B., Cooper, M., Guarino Reid, L., Vanderheiden, G. (eds.): Web Content Accessibility Guidelines 2.0, W3C Candidate Recommendation, April 30 (2008),

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Dimitris Spiliotopoulos
    • 1
  • Panagiota Stavropoulou
    • 1
  • Georgios Kouroupetroglou
    • 1
  1. 1.Department of Informatics and TelecommunicationsNational and Kapodistrian University of AthensAthensGreece

Personalised recommendations