The Effect of Multimodal Feedback Presented via a Touch Screen on the Performance of Older Adults

  • Ju-Hwan Lee
  • Ellen Poliakoff
  • Charles Spence
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5763)


Many IT devices – such as mobile phones and PDAs – have recently started to incorporate easy-to-use touch screens. There is an associated need for more effective user interfaces for touch screen devices that have a small screen area. One attempt to make such devices more effective and/or easy to use has come through the introduction of multimodal feedback from two or more sensory modalities. Multimodal feedback might provide even larger benefits to older adults who are often unfamiliar with recent developments in electronic devices, and may be suffering from the age-related degeneration of both cognitive and motor processes. Therefore, the beneficial effects associated with the use of multimodal feedback might be expected to be larger for older adults in perceptually and/or cognitively demanding situations. In the present study, we examined the potential benefits associated with the provision of multimodal feedback via a touch screen on older adults’ performance in a demanding dual-task situation. We compared unimodal (visual) feedback with various combinations of multimodal (bimodal and trimodal) feedback. We also investigated the subjective difficulty of the task as a function of the type of feedback provided in order to evaluate qualitative usability issues. Overall, the results demonstrate that the presentation of multimodal feedback with auditory signals via a touch screen device results in enhanced performance and subjective benefits for older adults.


Multimodal User Interface Multimodal Feedback Multimodal Interaction Older Adults Touch Screen 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Rogers, W.A.: Individual differences, aging, and human factors: An overview. In: Fisk, A.D., Rogers, W.A. (eds.) Handbook of Human Factors and the Older Adult, pp. 151–170. Academic Press, London (1997)Google Scholar
  2. 2.
    Martin, G.L.: Configuring a numeric keypad for a touch screen. Ergonomics 31, 945–953 (1988)CrossRefGoogle Scholar
  3. 3.
    Colle, H.A., Hiszem, K.J.: Standing at a kiosk: Effects of key size and spacing on touch screen numeric keypad performance and user preference. Ergonomics 47(13), 1406–1423 (2004)CrossRefGoogle Scholar
  4. 4.
    Gaver, W.: The SonicFinder: An interface that uses auditory icons. Human Computer Interaction 4(1), 67–94 (1989)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Akamatsu, M., Sato, S.: A multi-modal mouse with tactile and force feedback. Int. J. Hu.-Com. St. 40, 443–453 (1994)CrossRefGoogle Scholar
  6. 6.
    Vitense, H.S., Jacko, J.A., Emery, V.K.: Multimodal feedback: an assessment of performance and mental workload. Ergonomics 46, 68–87 (2003)CrossRefGoogle Scholar
  7. 7.
    Hart, S., Staveland, L.: Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In: Hancock, P.A., Meshkati, N. (eds.) Human mental workload, pp. 239–250. North-Holland, Amsterdam (1988)Google Scholar
  8. 8.
    Townsend, J.T., Ashby, F.G.: Stochastic modelling of elementary psychological processes. Cambridge University Press, London (1983)MATHGoogle Scholar
  9. 9.
    Jacko, J.A., Emery, V.K., Edwards, P.J., Ashok, M., Barnard, L., Kongnakorn, T., Moloney, K.P., Sainfort, F.: The effects of multimodal feedback on older adults’ task performance given varying levels of computer experience. Behav. Inform. Technol. 23(4), 247–264 (2004)CrossRefGoogle Scholar
  10. 10.
    Ketcham, C.J., Seidler, R.D., Van Gemmert, A.W., Stelmach, G.E.: Age-related kinemetic differences as influenced by task difficulty, target size, and movement amplitude. J. Gerontol.: Psych. Sci. 57B(1), 54–64 (2002)CrossRefGoogle Scholar
  11. 11.
    Poliakoff, E., Ashworth, S., Lowe, C., Spence, C.: Vision and touch in ageing: Crossmodal selective attention and visuotactile spatial interactions. Neuropsychologia 44, 507–517 (2006)CrossRefGoogle Scholar
  12. 12.
    Laurienti, P.J., Burdette, J.H., Maldjian, J.A., Wallace, M.T.: Enhanced multisensory integration in older adults. Neurobiol. Aging 27, 1155–1163 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Ju-Hwan Lee
    • 1
  • Ellen Poliakoff
    • 2
  • Charles Spence
    • 1
  1. 1.Crossmodal Research Laboratory, Department of Experimental PsychologyUniversity of OxfordOxfordUK
  2. 2.School of Psychological ScienceUniversity of ManchesterManchesterUK

Personalised recommendations