Universal Access in the Information Society

, Volume 17, Issue 2, pp 257–274 | Cite as

Interactivity and multimodality in language learning: the untapped potential of audiobooks

  • Emanuela Marchetti
  • Andrea Valente
Long Paper


In this work, we present three case studies, involving classes in primary and secondary schools, in Denmark. The studies, conducted in the past 2 years, show how audio content can be generated and shared among teachers and learners, how audio material can be made more interactive to offer fruition similar to that of digital games, and how language learning can benefit from adding a social dimension to audiobooks. All case studies were conducted in a user-centered fashion and build on social semiotics, in which interactive audiobooks are seen as providing new ways to receive, interpret, and share literary texts. Local primary and secondary schools were involved in ethnographic user studies and qualitative evaluations with semi-functioning prototypes. In the main case study presented, social interaction was chosen as key feature to allow high-school students and teachers to annotate audiobooks, then share and comment on the annotations; the social context in this case is a digitally augmented English teaching class. To better investigate the potential of sharable audiobook annotations, we also created a mockup supporting the workflow of the main case study, using standard YouTube annotations and freely available audiobooks. The findings and technical solutions explored in the three studies are the basis for design guidelines aiming at making audiobooks interactive and better integrated in learning contexts.


e-Learning Multimodal interaction Information presentation Knowledge management 


  1. 1.
    Adkins, D., Bushman, B.: A special needs approach. A study of how libraries can start programs for children with disabilities. Child. Libr. 13(3), 1–33 (2015). doi: 10.5860/cal.13n3.28 Google Scholar
  2. 2.
    Alcantud Díaz, M., Gregori-Signes, C.: Audiobooks: improving fluency and instilling literary skills and education for development. Tejuelo 20, 111–125 (2014)Google Scholar
  3. 3.
    Baskin, B.H., Harris, K.: Heard any good books lately? The case for audio books in the secondary classroom. J Read. 38(5), 372–376 (1995).
  4. 4.
    Clarkson, P.J., Coleman, R.: History of inclusive design in the UK. Appl. Ergon. XXX, 1–13 (2016). doi: 10.1016/j.apergo.2013.03.002 Google Scholar
  5. 5.
    de Verdier, K., Ek, U.: A longitudinal study of reading development, academic achievement, and support in Swedish inclusive education for students with blindness of severe visual impairment. J Vis Impair Blind 108(6), 130–140 (2014)Google Scholar
  6. 6.
    Design Council: Inclusive design education resource. Design Council, London, UK (2008).
  7. 7.
    Furini, M.: Beyond passive audiobooks: how digital audiobooks get interactive. In: IEEE Consumer Communication and Networking, IEEE Press, New York, pp. 971–975 (2007). doi: 10.1109/CCNC.2007.196
  8. 8.
    Gardner, H.: Multiple Intelligences. Basic Books, New York (2006)Google Scholar
  9. 9.
    Hattie, J., Gan, M.: Instruction based on feedback. In: Mayer, R.E., Alexander, P.A. (eds.) Handbook of research on learning and instruction, pp. 249–271. Routledge, New York (2011)Google Scholar
  10. 10.
    Huber, C., Röber, N., Hartmann, K., Masuch, M.: Evolution of interactive audiobooks. In: 2nd Conference on Interaction with Sound (Audio Mostly 2007), Fraunhofer Institute for Digital Media Technology IDMT, pp. 166–167 (2007)Google Scholar
  11. 11.
    Jordan, B., Henderson, A.: Interaction analysis. Foundations and practice. J Leav Sci, Erlbaum Associates Inc., 4(1), 39–103 (1995).
  12. 12.
    Kress, G.: Multimodality: a social semiotic approach to contemporary communication. Routledge, London (2010)Google Scholar
  13. 13.
    Kress, G., van Leeuwen, T.: Reading Images, The Grammar of Visual Design. Routledge, London (2006)Google Scholar
  14. 14.
    Landa, R.: Essential Graphic Design Solutions. Wadsworth Censage Learning (2014)Google Scholar
  15. 15.
    Marchetti, E., Petersson Brooks, E.: From lecturing to apprenticeship. In: Fourth International Conference on Mobile, Hybrid, and On-line Learning, IARIA, pp. 225–224 (2012)Google Scholar
  16. 16.
    Persson, H., Åhman, H., Arvei, A., Gulliksen, J.: Universal design, inclusive design, accessible design, design for all: different concepts-one goal? On the concept of accessibility-historical, methodological and philosophical aspects. Univ. Access Inf. Soc. 14, 505–526 (2015). doi: 10.1007/s10209-014-0358-z CrossRefGoogle Scholar
  17. 17.
    Pink, S.: Doing Visual Ethnography. Sage, Los Angeles, London (2007)CrossRefGoogle Scholar
  18. 18.
    Prieto, L.P., Sharma, K., Dillenbourg, P.: Studying teacher orchestration load in technology-enhanced classrooms: a mixed-method approach and case study. In: The 10th European Conference on Technology-Enhanced Learning (EC-TEL 2015), Springer Switzerland, pp. 1–14 (2015). doi: 10.1007/978-3-319-24258-3_20
  19. 19.
    Rogoff, B.: Apprenticeship in Thinking. Cognitive Development in Social Context. Oxford University Press, Oxford (1990)Google Scholar
  20. 20.
    van Zeijl, M.: The Soundwalker in the Street: Location-Based Audio Walks and the Poetic Re-imagination of Space. Arts and Technology, pp. 17–24. Springer, Berlin (2013). doi: 10.1007/978-3-642-37982-6_3
  21. 21.
    Waller, S., Bradley, M., Hosking, I., Clarkson, J.: Making the case for inclusive design. Appl. Ergon. 46, 235–247 (2015). doi: 10.1016/j.apergo.2013.03.012 CrossRefGoogle Scholar
  22. 22.
    Wilde S, Larsson J (2007) Listen! It’s Good for Kids. AudioFile, pp. 23–25.
  23. 23.
    Yliriksu, S., Buur, J.: Designing with Video. Focusing the User Centred Design Process. Springer, London (2007). doi: 10.1007/978-1-84628-961-3 Google Scholar
  24. 24.
    Zimmerman, J., Forlizzi, J., Evenson, S.: Research Through Design as a Method for Interaction Design Research in HCI. Hum. Comput. Inter. Inst. Pap. 41, 1–11 (2007). doi: 10.1145/1240624.1240704 Google Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  1. 1.Department for the Study of Culture, MediaUniversity of Southern Denmark (SDU)OdenseDenmark
  2. 2.Maersk Mc-Kinney Moller InstituteUniversity of Southern Denmark (SDU)OdenseDenmark

Personalised recommendations