EMA-Tactons: Vibrotactile External Memory Aids in an Auditory Display

  • Johan Kildal
  • Stephen A. Brewster
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4663)


Exploring any new data set always starts with gathering overview information. When this process is done non-visually, interactive sonification techniques have proved to be effective and efficient ways of getting overview information, particularly for users who are blind or visually impaired. Under certain conditions, however, the process of data analysis cannot be completed due to saturation of the user’s working memory. This paper introduces EMA-Tactons, vibrotactile external memory aids that are intended to support working memory during the process of data analysis, combining vibrotactile and audio stimuli in a multimodal interface. An iterative process led to a design that significantly improves the performance (in terms of effectiveness) of users solving complex data explorations. The results provide information about the convenience of using EMA-Tactons with other auditory displays, and the iterative design process illustrates the challenges of designing multimodal interaction techniques.


vibrotactile external memory aid overview visual impairment high-density sonification 


  1. 1.
    Shneiderman, B.: The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations. In: IEEE Symposium on Visual Languages, Boulder, CO, USA, pp. 336–343. IEEE Comp. Soc. Press, Los Alamitos (1996)Google Scholar
  2. 2.
    Zhao, H., Plaisant, C., Shneiderman, B., Duraiswami, R.: Sonification of Geo-Referenced Data for Auditory Information Seeking: Design Principle and Pilot Study. In: Int. Conf. Auditory Display, Sydney, Australia (2004)Google Scholar
  3. 3.
    Kildal, J., Brewster, S.: Exploratory Strategies and Procedures to Obtain Non-Visual Overviews Using Tablevis. Int. J. Disabil. Human Dev. 5(3), 285–294 (2006)CrossRefGoogle Scholar
  4. 4.
    Flowers, J.H.: Thirteen Years of Reflection on Auditory Graphing: Promises, Pitfalls, and Potential New Directions. In: Int. Symposium Auditory Graphs. Int. Conf. Auditory Display, Limerick, Ireland (2005)Google Scholar
  5. 5.
    Kildal, J., Brewster, S.: Providing a Size-Independent Overview of Non-Visual Tables. In: Int. Conf. Auditory Display, Queen Mary, University of London, pp. 8–15 (2006)Google Scholar
  6. 6.
    Kwasnik, B.H.: A Descriptive Study of the Functional Components of Browsing. In: Ifip Tc2/Wg2.7 Working Conference on Engineering for Human-Computer Interaction, pp. 191–203. North-Holland, Amsterdam (1992)Google Scholar
  7. 7.
    Miller, G.A.: The Magical Number Seven Plus or Minus Two: Some Limits on Our Capacity for Processing Information. The Psychological Review 63, 81–97 (1956)CrossRefGoogle Scholar
  8. 8.
    Wall, S., Brewster, S.: Providing External Memory Aids in Haptic Visualisations for Blind Computer Users. Int. J. Disabil. Human Dev. 4(3), 285–294 (2006)Google Scholar
  9. 9.
    Brewster, S., Brown, L.: Tactons: Structured Tactile Messages for Non-Visual Information Display. In: Australasian User Interface Conf., pp. 15–23. Australian Comp. Soc., Dunedin, New Zealand (2004)Google Scholar
  10. 10.
    Lee, J.C., Dietz, P.H., Leigh, D., Yerazunis, W.S., Hudson, S.E.: Haptic Pen: A Tactile Feedback Stylus for Touch Screens. In: Annual ACM Symposium on User Interface Software and Technology, pp. 291–294. ACM Press, New York (2004)Google Scholar
  11. 11.
    Hart, S., Wickens, C.: Workload Assessment and Prediction. Manprint, an Approach to Systems Integration. In: Booher, H.R.(ed.) Van Nostrand Reinhold, pp. 257–296 (1990) Google Scholar
  12. 12.
    Spence, C., Driver, J.: Crossmodal Space and Crossmodal Attention. Oxford University Press, Oxford (2004)CrossRefGoogle Scholar
  13. 13.
    Bregman, A.: Auditory Scene Analysis: The Perceptual Organization of Sound. The MIT Press, Cambridge (1994)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2007

Authors and Affiliations

  • Johan Kildal
    • 1
  • Stephen A. Brewster
    • 1
  1. 1.Glasgow Interactive Systems Group, Department of Computing Science, University of Glasgow. Glasgow, G12 8QQUK

Personalised recommendations