Experimental Brain Research

, Volume 234, Issue 5, pp 1199–1207 | Cite as

School-aged children can benefit from audiovisual semantic congruency during memory encoding

  • Jenni HeikkiläEmail author
  • Kaisa Tiippana
Research Article


Although we live in a multisensory world, children’s memory has been usually studied concentrating on only one sensory modality at a time. In this study, we investigated how audiovisual encoding affects recognition memory. Children (n = 114) from three age groups (8, 10 and 12 years) memorized auditory or visual stimuli presented with a semantically congruent, incongruent or non-semantic stimulus in the other modality during encoding. Subsequent recognition memory performance was better for auditory or visual stimuli initially presented together with a semantically congruent stimulus in the other modality than for stimuli accompanied by a non-semantic stimulus in the other modality. This congruency effect was observed for pictures presented with sounds, for sounds presented with pictures, for spoken words presented with pictures and for written words presented with spoken words. The present results show that semantically congruent multisensory experiences during encoding can improve memory performance in school-aged children.


Audiovisual Children Development Memory Multisensory Semantic congruency 



This research was funded by a grant from the University of Helsinki. It is part of the research activities of CICERO Learning Network, Finland, We are grateful to the pupils, guardians and teachers in Iivisniemi elementary school and Tähtiniitty elementary school, Espoo, Finland, where the research was conducted. We thank Professor Kimmo Alho for comments on the manuscript. The Multimodal Stimulus Set was developed by T.R. Schneider, S. Debener and A.K. Engel at the Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-Eppendorf, Germany.


  1. Bahrick L, Lickliter R (2012) The role of intersensory redundancy in early perceptual, cognitive and social development. In: Bremner A, Lewkowich D, Spence C (eds) Multisensory development. Oxford University Press, UK, pp 183–206CrossRefGoogle Scholar
  2. Brandwein A, Foxe J, Russo N, Altschuler T, Gomes H, Molholm S (2011) The development of audiovisual multisensory integration across childhood and early adolescence: a high-density electrical mapping study. Cereb Cortex 21:1042–1055CrossRefPubMedPubMedCentralGoogle Scholar
  3. Calvert G, Spence C, Stein B (2004) The Handbook of Multisensory Processes. The MIT Press, LondonGoogle Scholar
  4. Chen Y, Spence C (2010) When hearing the bark helps to identify the dog: semantically-congruent sounds modulate the identification of masked pictures. Cognition 114:389–404CrossRefPubMedGoogle Scholar
  5. Colavita F (1974) Human sensory dominance. Percept Psychophys 25:345–347CrossRefGoogle Scholar
  6. Constantidou F, Danos M, Nelson D, Baker S (2011) Effects of modality presentation of working memory in school-age children: evidence for the pictorial superiority hypothesis. Child Neuropsychol 17:173–196CrossRefGoogle Scholar
  7. Dekker TM, Mareschal D, Johnson MH, Sereno MI (2014) Picturing words? Sensorimotor cortex activation for printed words in child and adult readers. Brain Lang 139:58–67CrossRefPubMedPubMedCentralGoogle Scholar
  8. Ernst M (2008) Multisensory integration: a late bloomer. Curr Biol 18:519–521CrossRefGoogle Scholar
  9. Ernst MO, Bülthoff HH (2004) Merging the senses into a robust percept. Trends Cogn Sci 8:162–169CrossRefPubMedGoogle Scholar
  10. Gori M, Del Viva M, Sandini G, Burr D (2008) Young children do not integrate visual and haptic information. Curr Biol 18:689–693CrossRefGoogle Scholar
  11. Heikkilä J, Alho K, Hyvönen H, Tiippana K (2015) Audiovisual semantic congruency during encoding enhances memory performance. Exp Psychol 62:123–130CrossRefPubMedGoogle Scholar
  12. Hillock A, Powers A, Wallace M (2011) Binding sights and sounds: age-related changes in multisensory temporal processing. Neuropsychologia 49:461–467CrossRefPubMedPubMedCentralGoogle Scholar
  13. Jordan K, Baker J (2011) Multisensory information boosts numerical matching abilities in young children. Dev Sci 14:205–213CrossRefPubMedGoogle Scholar
  14. Laurienti PJ, Kraft RA, Maldjian JA, Burdette JH, Wallace MT (2004) Semantic congruence is a critical factor in multisensory behavioral performance. Exp Brain Res 158:405–414CrossRefPubMedGoogle Scholar
  15. Lehmann S, Murray MM (2005) The role of multisensory memories in unisensory object discrimination. Cogn Brain Res 24:326–334CrossRefGoogle Scholar
  16. Lewkowicz DJ, Ghazanfar AA (2009) The emergence of multisensory systems through perceptual narrowing. Trends Cogn Sci 13:470–478CrossRefPubMedGoogle Scholar
  17. Molholm S, Ritter W, Javitt DC, Foxe JJ (2004) Multisensory visual–auditory object recognition in humans: a high-density electrical mapping study. Cereb Cortex 14:452–465CrossRefPubMedGoogle Scholar
  18. Moran Z, Bachman P, Pham P, Cho S, Cannon T, Shams L (2013) Multisensory encoding improves auditory recognition. Multisens Res 26:581–592CrossRefPubMedGoogle Scholar
  19. Murray MM, Michel CM, Grave de Peralta R, Ortigue S, Brunet D, Gonzalez Andino S, Schnider A (2004) Rapid discrimination of visual and multisensory memories revealed by electrical neuroimaging. Neuroimage 21:125–135CrossRefPubMedGoogle Scholar
  20. Nava E, Pavani F (2013) Changes in sensory dominance during childhood: converging evidence from the Colavita effect and the sound-induced flash illusion. Child Dev 84:604–616CrossRefPubMedGoogle Scholar
  21. Ngo M, Sinnet S, Soto-Faraco S, Spence C (2010) Repetition blindness and the Colavita effect. Neurosci Lett 480:186–190CrossRefPubMedGoogle Scholar
  22. Potter MC (1993) Very short-term conceptual memory. Mem Cogn 21:156–161CrossRefGoogle Scholar
  23. Potter MC (2012) Conceptual short term memory in perception and thought. Front Psychol 3:1–11Google Scholar
  24. Schneider TR, Engel AK, Debener S (2008) Multisensory identification of natural objects in a two-way crossmodal priming paradigm. Exp Psychol 55:121–132CrossRefPubMedGoogle Scholar
  25. Shams L, Seitz AR (2008) Benefits of multisensory learning. Trends Cogn Sci 12:411–417CrossRefPubMedGoogle Scholar
  26. Thelen A, Talsma D, Murray M (2015) Single-trial multisensory memories affect later auditory and visual object discrimination. Cognition 138:148–160CrossRefPubMedGoogle Scholar
  27. Torppa M, Lyytinen P, Erskine J, Eklund K, Lyytinen H (2010) Language development, literacy skills and predictive connections to reading in Finnish children with and without familiar risk for dyslexia. J Learn Disabil 43:308–321CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Division of Cognitive Psychology and Neuropsychology, Institute of Behavioural SciencesUniversity of HelsinkiHelsinkiFinland

Personalised recommendations