Learning into the Wild: A Protocol for the Use of 360° Video for Foreign Language Learning

  • Claudia RepettoEmail author
  • Serena Germagnoli
  • Stefano Triberti
  • Giuseppe Riva
Conference paper
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 253)


Learning a second language could be a boring task if accomplished by repeating bilingual words lists. Laboratory research demonstrated that second language learning is more efficient if the material is enriched by pictures or gestures during the encoding phase. Here we want to test the impact of 360° videos on foreign language learning. The 360° videos are spherical videos that allow a lifelike exploration of the environment, if experienced immersively (namely, by means of a Head Mounted Display). The protocol includes ten 360° videos representing natural landscapes, sport performances and adventures; each video has been enriched with a narrative that guides the subject’s attention towards the relevant elements named in English (the second language). The goals are twofold: first, we want to investigate whether the videos fruition is able to promote language learning; second, we want to verify the acceptability and perceived utility of this technology by potential users.


Virtual Reality 360° video Second language learning Embodied Cognition 



This work was partially supported by Fondazione Cassa di Risparmio di Alessandria.


  1. 1.
    Fodor, J.A.: The Modularity of Mind. MIT press, Cambridge (1983)Google Scholar
  2. 2.
    Fodor, J.A.: The Language of Thought. Harvard University Press, Cambridge (1975)Google Scholar
  3. 3.
    Barsalou, L.W.: Grounded cognition. Annu. Rev. Psychol. 59, 617–645 (2008)CrossRefGoogle Scholar
  4. 4.
    Shapiro, L.A.: Embodied Cognition. Routledge, Abingdon (2011)Google Scholar
  5. 5.
    Martin, A., Haxby, J.V., Lalonde, F.M., Wiggs, C.L., Ungerleider, L.G.: Discrete cortical regions associated with knowledge of color and knowledge of action. Science (80-) 270, 102–105 (1995)CrossRefGoogle Scholar
  6. 6.
    Goldberg, R.F., Perfetti, C.A., Schneider, W.: Perceptual knowledge retrieval activates sensory brain regions. J. Neurosci. 26, 4917–4921 (2006)CrossRefGoogle Scholar
  7. 7.
    Thompson-Schill, S.L.: Neuroimaging studies of semantic memory: inferring ‘how’ from ‘where’. Neuropsychologia 41, 280–292 (2003)CrossRefGoogle Scholar
  8. 8.
    Wilson, M.: Six views of embodied cognition. Psychon. Bull. Rev. 9, 625–636 (2002)CrossRefGoogle Scholar
  9. 9.
    Glenberg, A.M.: What memory is for. Behav. Brain Sci. 20, 1–55 (1997)Google Scholar
  10. 10.
    Galantucci, B., Fowler, C.A., Turvey, M.T.: The motor theory of speech perception reviewed. Psychon. Bull. Rev. 13, 361–377 (2006)CrossRefGoogle Scholar
  11. 11.
    Liberman, A.M., Mattingly, I.G.: The motor theory of speech perception. Cognition 21, 1–36 (1985)CrossRefGoogle Scholar
  12. 12.
    Glenberg, A.M., Kaschak, M.P.: Grounding language in action. Psychon. Bull. Rev. 9, 558–565 (2002)CrossRefGoogle Scholar
  13. 13.
    Glenberg, A.M., Gutierrez, T., Levin, J.R., Japuntich, S., Kaschak, M.P.: Activity and imagined activity can enhance young children’s reading comprehension. J. Educ. Psychol. 96, 424–436 (2004)CrossRefGoogle Scholar
  14. 14.
    Glenberg, A.M., Brown, M., Levin, J.R.: Enhancing comprehension in small reading groups using a manipulation strategy. Contemp. Educ. Psychol. 32, 389–399 (2007)CrossRefGoogle Scholar
  15. 15.
    Marley, S.C., Levin, J.R., Glenberg, A.M.: Improving Native American children’s listening comprehension through concrete representations. Contemp. Educ. Psychol. 32, 537–550 (2007)CrossRefGoogle Scholar
  16. 16.
    Glenberg, A.M., Goldberg, A.B., Zhu, X.: Improving early reading comprehension using embodied CAI. Instr. Sci. 39, 27–39 (2011)CrossRefGoogle Scholar
  17. 17.
    Shams, L., Seitz, A.R.: Benefits of multisensory learning. Trends Cogn. Sci. 12, 411–417 (2008)CrossRefGoogle Scholar
  18. 18.
    Paivio, A., Csapo, K.: Concrete image and verbal memory codes. J. Exp. Psychol. 80, 279 (1969)CrossRefGoogle Scholar
  19. 19.
    Craik, F.I.M., Tulving, E.: Depth of processing and the retention of words in episodic memory. J. Exp. Psychol. Gen. 104, 268–294 (1975)CrossRefGoogle Scholar
  20. 20.
    Macedonia, M., von Kriegstein, K.: Gestures enhance foreign language learning. Biolinguistics 6, 393–416 (2012)Google Scholar
  21. 21.
    Zimmer, H.D., et al.: Memory for Action: A Distinct for of Episodic Memory?. Oxford University Press, Oxford (2001)Google Scholar
  22. 22.
    Nilsson, L.G.: Remembering actions and words. In: Tulving, E., Craik, F.I.M. (eds.) The Oxford Handbook of Memory, pp. 137–148. Oxford University Press (2000)Google Scholar
  23. 23.
    Engelkamp, J.: Memory for Actions. Psychology Press/Taylor & Francis, Hove/Abingdon (1998)Google Scholar
  24. 24.
    Tellier, M.: The effect of gestures on second language memorisation by young children. Gesture 8, 219–235 (2008)CrossRefGoogle Scholar
  25. 25.
    Macedonia, M., Knösche, T.R.: Body in mind: how gestures empower foreign language learning. Mind Brain Educ. 5, 196–211 (2011)CrossRefGoogle Scholar
  26. 26.
    Mayer, K.M., Yildiz, I.B., Macedonia, M., Von Kriegstein, K.: Visual and motor cortices differentially support the translation of foreign language words. Curr. Biol. 25, 530–535 (2015)CrossRefGoogle Scholar
  27. 27.
    Triberti, S., Riva, G.: Being present in action: a theoretical model about the ‘interlocking’ between intentions and environmental affordances. Front. Psychol. 6, 2052 (2016)CrossRefGoogle Scholar
  28. 28.
    Schwienhorst, K.: The state of VR: a meta-analysis of virtual reality tools in second language acquisition. Comput. Assist. Lang. Learn. 15, 221–239 (2002)CrossRefGoogle Scholar
  29. 29.
    Rose, H., Billinghurst, M.: Zengo Sayu: an immersive educational environment for learning Japanese (1995)Google Scholar
  30. 30.
    Amoia, M., et al.: A serious game for second language acquisition in a virtual environment. J. Syst. Cybern. Inform. 10, 24–34 (2012)Google Scholar
  31. 31.
    Bertinetto, P., et al.: Corpus e Lessico di Frequenza dell’Italiano Scritto (CoLFIS) (2005).
  32. 32.
    Davis, F.D.: Perceived usefulness, perceived ease of use and user acceptance information technology. MIS Q. 13, 319–339 (1989)CrossRefGoogle Scholar
  33. 33.
    IJsselsteijn, W., de Riddera, H., Hamberga, R., Bouwhuisa, D., Freeman, J.: Perceived depth and the feeling of presence in 3DTV. Displays 18, 207–214 (1998)CrossRefGoogle Scholar

Copyright information

© ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2018

Authors and Affiliations

  • Claudia Repetto
    • 1
    Email author
  • Serena Germagnoli
    • 1
  • Stefano Triberti
    • 1
  • Giuseppe Riva
    • 1
    • 2
  1. 1.Department of PsychologyCatholic University of Sacred HeartMilanItaly
  2. 2.Applied Technology for Neuropsychology LaboratoryIRCCS Istituto Auxologico ItalianoMilanItaly

Personalised recommendations