Skip to main content

Interactivity and multimodality in language learning: the untapped potential of audiobooks

Universal Access in the Information Society volume 17, pages 257–274 (2018)Cite this article

Abstract

In this work, we present three case studies, involving classes in primary and secondary schools, in Denmark. The studies, conducted in the past 2 years, show how audio content can be generated and shared among teachers and learners, how audio material can be made more interactive to offer fruition similar to that of digital games, and how language learning can benefit from adding a social dimension to audiobooks. All case studies were conducted in a user-centered fashion and build on social semiotics, in which interactive audiobooks are seen as providing new ways to receive, interpret, and share literary texts. Local primary and secondary schools were involved in ethnographic user studies and qualitative evaluations with semi-functioning prototypes. In the main case study presented, social interaction was chosen as key feature to allow high-school students and teachers to annotate audiobooks, then share and comment on the annotations; the social context in this case is a digitally augmented English teaching class. To better investigate the potential of sharable audiobook annotations, we also created a mockup supporting the workflow of the main case study, using standard YouTube annotations and freely available audiobooks. The findings and technical solutions explored in the three studies are the basis for design guidelines aiming at making audiobooks interactive and better integrated in learning contexts.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Notes

  1. Danish primary school site: https://www.folkeskolen.dk/504367/det-skal-vaere-nemmere-for-laerere-at-finde-relevante-e--og-lydboeger (last seen on the 23th of March 2017).

  2. Freely available on GooglePlay.

  3. List of popular audio walk on SoundCloud: https://soundcloud.com/tags/audio%20walk (last seen on the March 23, 2017).

  4. AppInventor’s official page: http://appinventor.mit.edu/explore/ (last seen on the March 23, 2017).

  5. Unity3d Web site: https://unity3d.com/ (last seen on the March 23, 2017).

  6. Kaizena’s Web page: https://kaizena.com/ (last seen on the March 23, 2017).

  7. A copy of Frankenstein is freely available at: http://www.gutenberg.org/ebooks/20038 (last seen on the March 23, 2017).

  8. The definition of the ID3v2 standard can be found at: http://id3.org/id3v2-chapters-1.0 (last seen on the March 23, 2017).

References

  1. Adkins, D., Bushman, B.: A special needs approach. A study of how libraries can start programs for children with disabilities. Child. Libr. 13(3), 1–33 (2015). doi:10.5860/cal.13n3.28

    Google Scholar 

  2. Alcantud Díaz, M., Gregori-Signes, C.: Audiobooks: improving fluency and instilling literary skills and education for development. Tejuelo 20, 111–125 (2014)

    Google Scholar 

  3. Baskin, B.H., Harris, K.: Heard any good books lately? The case for audio books in the secondary classroom. J Read. 38(5), 372–376 (1995). http://www.jstor.org/stable/40033253

  4. Clarkson, P.J., Coleman, R.: History of inclusive design in the UK. Appl. Ergon. XXX, 1–13 (2016). doi:10.1016/j.apergo.2013.03.002

    Google Scholar 

  5. de Verdier, K., Ek, U.: A longitudinal study of reading development, academic achievement, and support in Swedish inclusive education for students with blindness of severe visual impairment. J Vis Impair Blind 108(6), 130–140 (2014)

    Google Scholar 

  6. Design Council: Inclusive design education resource. Design Council, London, UK (2008). http://www.designcouncil.org.uk/inclusive-design-education-resource

  7. Furini, M.: Beyond passive audiobooks: how digital audiobooks get interactive. In: IEEE Consumer Communication and Networking, IEEE Press, New York, pp. 971–975 (2007). doi:10.1109/CCNC.2007.196

  8. Gardner, H.: Multiple Intelligences. Basic Books, New York (2006)

    Google Scholar 

  9. Hattie, J., Gan, M.: Instruction based on feedback. In: Mayer, R.E., Alexander, P.A. (eds.) Handbook of research on learning and instruction, pp. 249–271. Routledge, New York (2011)

    Google Scholar 

  10. Huber, C., Röber, N., Hartmann, K., Masuch, M.: Evolution of interactive audiobooks. In: 2nd Conference on Interaction with Sound (Audio Mostly 2007), Fraunhofer Institute for Digital Media Technology IDMT, pp. 166–167 (2007)

  11. Jordan, B., Henderson, A.: Interaction analysis. Foundations and practice. J Leav Sci, Erlbaum Associates Inc., 4(1), 39–103 (1995). http://www.jstor.org/stable/1466849

  12. Kress, G.: Multimodality: a social semiotic approach to contemporary communication. Routledge, London (2010)

    Google Scholar 

  13. Kress, G., van Leeuwen, T.: Reading Images, The Grammar of Visual Design. Routledge, London (2006)

    Google Scholar 

  14. Landa, R.: Essential Graphic Design Solutions. Wadsworth Censage Learning (2014)

  15. Marchetti, E., Petersson Brooks, E.: From lecturing to apprenticeship. In: Fourth International Conference on Mobile, Hybrid, and On-line Learning, IARIA, pp. 225–224 (2012)

  16. Persson, H., Åhman, H., Arvei, A., Gulliksen, J.: Universal design, inclusive design, accessible design, design for all: different concepts-one goal? On the concept of accessibility-historical, methodological and philosophical aspects. Univ. Access Inf. Soc. 14, 505–526 (2015). doi:10.1007/s10209-014-0358-z

    Article  Google Scholar 

  17. Pink, S.: Doing Visual Ethnography. Sage, Los Angeles, London (2007)

    Book  Google Scholar 

  18. Prieto, L.P., Sharma, K., Dillenbourg, P.: Studying teacher orchestration load in technology-enhanced classrooms: a mixed-method approach and case study. In: The 10th European Conference on Technology-Enhanced Learning (EC-TEL 2015), Springer Switzerland, pp. 1–14 (2015). doi:10.1007/978-3-319-24258-3_20

  19. Rogoff, B.: Apprenticeship in Thinking. Cognitive Development in Social Context. Oxford University Press, Oxford (1990)

    Google Scholar 

  20. van Zeijl, M.: The Soundwalker in the Street: Location-Based Audio Walks and the Poetic Re-imagination of Space. Arts and Technology, pp. 17–24. Springer, Berlin (2013). doi:10.1007/978-3-642-37982-6_3

  21. Waller, S., Bradley, M., Hosking, I., Clarkson, J.: Making the case for inclusive design. Appl. Ergon. 46, 235–247 (2015). doi:10.1016/j.apergo.2013.03.012

    Article  Google Scholar 

  22. Wilde S, Larsson J (2007) Listen! It’s Good for Kids. AudioFile, pp. 23–25. http://www.audiofilemagazine.com/content/uploaded/media/listen-goodforkids.pdf

  23. Yliriksu, S., Buur, J.: Designing with Video. Focusing the User Centred Design Process. Springer, London (2007). doi:10.1007/978-1-84628-961-3

    Google Scholar 

  24. Zimmerman, J., Forlizzi, J., Evenson, S.: Research Through Design as a Method for Interaction Design Research in HCI. Hum. Comput. Inter. Inst. Pap. 41, 1–11 (2007). doi:10.1145/1240624.1240704

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department for the Study of Culture, Media, University of Southern Denmark (SDU), Odense, Denmark

    Emanuela Marchetti

  2. Maersk Mc-Kinney Moller Institute, University of Southern Denmark (SDU), Odense, Denmark

    Andrea Valente

Authors
  1. Emanuela Marchetti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrea Valente
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Emanuela Marchetti.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Marchetti, E., Valente, A. Interactivity and multimodality in language learning: the untapped potential of audiobooks. Univ Access Inf Soc 17, 257–274 (2018). https://doi.org/10.1007/s10209-017-0549-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-017-0549-5

Keywords

  • e-Learning
  • Multimodal interaction
  • Information presentation
  • Knowledge management