Skip to main content

Advertisement

SpringerLink
Log in

Multimodal interactions in typically and atypically developing children: natural versus artificial environments

  • Review
  • Published:
Cognitive Processing Aims and scope Submit manuscript

Abstract

This review addresses the central role played by multimodal interactions in neurocognitive development. We first analyzed our studies of multimodal verbal and nonverbal cognition and emotional interactions within neuronal, that is, natural environments in typically developing children. We then tried to relate them to the topic of creating artificial environments using mobile toy robots to neurorehabilitate severely autistic children. By doing so, both neural/natural and artificial environments are considered as the basis of neuronal organization and reorganization. The common thread underlying the thinking behind this approach revolves around the brain’s intrinsic properties: neuroplasticity and the fact that the brain is neurodynamic. In our approach, neural organization and reorganization using natural or artificial environments aspires to bring computational perspectives into cognitive developmental neuroscience.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Adolphs R, Tranel D (1999) Intact recognition of emotional prosody following amygdala damage. Neuropsychologia 37:1285–1292

    Article  PubMed  CAS  Google Scholar 

  • Aggleton JP (2000) The amygdala: a functional analysis. Oxford University Press, Oxford

    Google Scholar 

  • Amedi A, Malach R, Hendler T, Peled S, Zohary E (2001) Visuo-haptic object-related activation in the ventral visual pathway. Nat Neurosci 4:324–330

    Article  PubMed  CAS  Google Scholar 

  • Banati RB, Goerres GW, Tjoa C, Aggleton JP, Grasby P (2000) The functional anatomy of visual-tactile integration in man: a study using positron emission tomography. Neuropsychologia 38(2):115–124

    Article  PubMed  CAS  Google Scholar 

  • Bates E (1999) Language and the infant brain. J Commun Disord 32:195–205

    Article  PubMed  CAS  Google Scholar 

  • Berthoz A (2003) La décision. Odile Jacob, Paris

    Google Scholar 

  • Billard A (2003) Robota: clever toy and educational tool. Rob Auton Syst 42:259–269

    Article  Google Scholar 

  • Billard A, Calinon S, Dillmann R, Schaal S (2008) Robot programming by demonstration. In: Siciliano B, Khatib O (eds) Handbook of robotics, (chapter 59). Springer, New York

    Google Scholar 

  • Bishop DVM (2006) What causes specific language impairment in children? Curr Dir Psychol Sci 15:217–221

    Article  PubMed  Google Scholar 

  • Bowler F (2012) Autism spectrum disorders (ASD). Autism 16(3):223–225

    Article  PubMed  Google Scholar 

  • Brass M, Heyes C (2005) Imitation: is cognitive neuroscience solving the correspondence problem? Trend Cogn Sci 9:489–495

    Article  Google Scholar 

  • Broman SH, Fletcher JM (1999) The changing nervous system. Neurobehavioral consequences of early brain disorders. Oxford University Press, Oxford

    Google Scholar 

  • Brothers L (1990) The social brain: a project for integrating primate behaviour and neurophysiology in a new domain. Concept Neurosci 1:27–51

    Google Scholar 

  • Callu D, Giannopulu I, Escolano S, Cusin F, Jacquier-Roux M, Dellatolas G (2005) Smooth pursuit eye movements are associated to phonological awareness in preschool children. Brain Cogn 58:217–225

    Article  PubMed  CAS  Google Scholar 

  • Cangelosi A (2010) Grounding language in action and perception: from cognitive agents to humanoid robots. Phys Life Rev 7:139–151

    Article  PubMed  Google Scholar 

  • Cappa SF, Perani D (2003) The neural correlates of noun and verb processing. J Neurolinguistics 16(2–3):183–189

    Article  Google Scholar 

  • Carpentier PA, Just MA, Keller T et al (2001) Dynamic cortical systems subserving cognition. fMRI studies with typical and atypical individuals. In: McClelland JL, Siegel RS (eds) Mechanisms of cognitive development. Lawrence Erlbaum, Associates Inc., Mahwah, pp 353–386

    Google Scholar 

  • Carr L, Iacoboni M, Dubeau MC, Mazziotta JC, Lenzi GL (2003) Neural mechanisms of empathy in humans: a relay from neural systems for imitation to limbic areas. Proc Natl Acad Sci USA 100:5497–5502

    Article  PubMed  CAS  Google Scholar 

  • Casey BJ, Tottenham N, Conor L, Durston S (2005) Imaging the developing brain: what have we learned about cognitive development? Trends Cogn Sci 9:104–110

    Article  PubMed  CAS  Google Scholar 

  • Chaminade T, Zecca M, Blakemore SL, Takanishi A, Frith CD, Micara S, Dario P, Rizzolatti G, Gallese V, Unilta MA (2010) Brain response to a humanoid robot in areas implicated in the perception of human emotional gestures. PLoS ONE 5(7):11577

    Article  Google Scholar 

  • Christiansen MH, Chater N (2008) Language as shaped by the brain. Behav Brain Sci 31:489–508

    PubMed  Google Scholar 

  • Corbett BA, Carmean V, Ravizza S, Wendelken C, Henry ML, Carter C, Rivera SM (2009) A functional and structural study of emotion and face processing in children with autism. Psychiatry Res 30:196–205

    Google Scholar 

  • Cusin F, Dellatolas G, Giannopulu I (2013) Action imitation and language embodiment in typically developing children aged 5 to 6 years old (submitted)

  • Dautenhahn K, Werry I (2002) A quantitative technique for analysing robot–human interactions. IEEE/RSJ international conference on intelligent robots and systems, 1132–1138

  • Dautenhahn K (2007) Socially intelligent robots: dimensions of human–robot interaction. Philosoph Trans R Soc B 362:679–704

    Article  Google Scholar 

  • Davis A, Robins BK, Dautenhahn K, Nehaniv CL, Powell S (eds). A comparison of interactive and robotic systems in therapy and education for children with autism. European conference for the advancement of assistive technology in Europe (AAATE’05) September 6–9 2005 IOS Press Lille, pp 353–357

  • Dehaene-Lambertz G, Dehaene S, Hertz-Pannier L (2002) Functional neuroimaging of speech perception in infants. Science 298:2013–2015

    Article  PubMed  CAS  Google Scholar 

  • Dehaene-Lambertz G, Hertz-Pannier L, Dubois J, Mériaux S, Roche A, Sigman M, Dehaene S (2006) Functional organization of perisylvian activation during presentation of sentences in preverbal infants. Proc Natl Acad Sci USA 103:14240–14245

    Article  PubMed  CAS  Google Scholar 

  • Dekker TM, Karmiloff-Smith A (2011) The dynamics of ontogeny. A neuroconstructivist perspective on genes, brains cognition and behavior. Prog Brain Res 189:23–33

    Article  PubMed  Google Scholar 

  • Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition Revised (DSM-IV-TR) (2003) Disorders usually first diagnosed in infancy, childhood, or adolescence, chapter 1

  • Duquette A, Michaud F, Mercier H (2004) Exploring the use of a mobile robot as an imitation agent with children with low-functioning autism. Autonomous Robots–Special Issue, 2, pp 147–157

  • Escalona A, Field T, Nadel J, Lundy B (2002) Brief report: imitation effects on children with autism. J Autism Dev Disord 32:141–144

    Article  PubMed  Google Scholar 

  • Fair DA, Cohen AL, Dosenbach NUF, Church JA, Miezin MM, Barch DM, Raichle ME, Petersen SE, Schlaggar BL (2008) The maturing architecture of the brain’s default network. Proc Natl Acad Sci USA 105:4028–4032

    Article  PubMed  CAS  Google Scholar 

  • Field TM, Woodson RW, Greenberg R, Cohen C (1982) Discrimination and imitation of facial expression by neonates. Science 218:179–181

    Article  PubMed  CAS  Google Scholar 

  • Frith U, Frith CD (2003) Development and neurophysiology of mentalizing. Philosoph Trans R Soc B Biol Sci 358:459–473

    Article  Google Scholar 

  • Frith U, Morton J, Leslie AM (1991) The cognitive basis of a biological disorder: autism. Trends Neurosci 14:433–438

    Article  PubMed  CAS  Google Scholar 

  • Garon N, Moore C (2004) Complex decision-making in early childhood. Brain Cogn 55:158–170

    Article  PubMed  Google Scholar 

  • Giannopulu I (2011a) Contribution à la compréhension des representations multimodales chez l’homme sain et chez des patients avec atteinte neuropsychologique: une perspective “life span”. Université Pierre et Marie Curie (Paris VI), Habilitation à Diriger des Recherches

    Google Scholar 

  • Giannopulu I (2011b) Cognitive and emotional interactions between autistic child, mobile toy robot and therapist. Front Comput Neurosci. doi:10.3389/conf.fncom.2011.52.00002

  • Giannopulu I (2012) From typical neurocognitive development to neurorehabilitation of autistic children using mobile toy robots. Robot and cognitive development conference, 10–12 September, Lausanne-Switzerland

  • Giannopulu I (2013) Multimodal human–robot interactions: the neurorehabilitation of severe autistic children. The Sixth International Conference on Advances in Computer-Human Interactions, IARIA, 24–27 February, Nice-France

  • Giannopulu I, Pradel G (2009a) Mobile toy robots can be used in autism therapy: an example of application. IEEE proceedings in the IROS 2009 paper number SuT8.pdf in the workshop CD

  • Giannopulu I, Pradel G (2009b) Interactions multimodales en situation de jeu libre entre enfants autistes et un robot mobile. 7ème Journée Nationale de la recherche en Robotique. Novembre, Domaine de La Grande Garenne 18330 Neuvy-sur-Barangeon

  • Giannopulu I, Pradel G (2010) Multimodal interactions in playful encounters of children with autism and a mobile robot. Neurorehabilitation 27(4):305–311

    PubMed  Google Scholar 

  • Giannopulu I, Pradel G (2012) From child–robot interaction to child–robot–therapist interaction: a case study in autism. Appl Bionics Biomech 9:173–179

    Google Scholar 

  • Giannopulu I, Sagot I (2010) Ressenti émotionnel positif dans une tâche expérimentale chez l’enfant. Annales Médico-Psychologiques 168(10):740–745

    Article  Google Scholar 

  • Giannopulu I, Cusin F, Escolano S, Dellatolas G (2008) Cognitive associations of bimanual haptico visual recognition in preschoolers. Child Neuropsychol 14:227–236

    Article  PubMed  CAS  Google Scholar 

  • Gogtay N, Giedd JN, Lusk L, Hayashi KM, Greenstein D, Vaituzis AC, Nugent TF 3rd, Herman DH, Clasen LS, Toga AW, Rapoport JL, Thompson PM (2004) Dynamic mapping of human cortical development during childhood through early adulthood. Proc Natl Acad Sci USA 101:8174–8179

    Article  PubMed  CAS  Google Scholar 

  • Grafton ST, Arbib MA, Fadiga L, Rizzolatti G (1996) Localisations of grasp representations in humans by positron emission tomography. 2. Observation compared with imagination. Exp Brain Res 112:103–111

    Article  PubMed  CAS  Google Scholar 

  • Hauk O, Johnsrude I, Pulvermüller F (2004) Somatotopic representation of action words in human motor and premotor cortex. Neuron 41(2):301–307

    Article  PubMed  CAS  Google Scholar 

  • Hay DF, Pedersen J, Nash A (1982) Dyadic interaction in the first year of life. In: Rubin KH, Ross HS (eds) Peer relationship and social skills in childhood. Springer, New York, pp 11–40

    Chapter  Google Scholar 

  • Hermsdörfer J, Goldenberg G, Wachsmuth C, Conrad B, Ceballos-Baumann AO, Bartenstein P, Schwaiger M, Boecker H (2001) Cortical correlates of gesture processing: clues to the cerebral mechanisms underlying apraxia during the imitation of meaningless gestures. Neuroimage 14:149–161

    Article  PubMed  Google Scholar 

  • Iacoboni M, Woods RP, Brass M, Bekkering H, Mazziotta JC, Rizzollati G (1999) Cortical mechanisms of human imitation. Science 286:2054–2528

    Article  Google Scholar 

  • Iacoboni M, Koski LM, Brass M, Bekkering H, Woods RP, Dubeau MC, Mazziota JC, Rizzolatti G (2001) Reafferent copies of imitated actions in the right superior temporal cortex. Proc Natl Acad Sci USA 98:13995–13999

    Article  PubMed  CAS  Google Scholar 

  • International Classification of diseases and related health problems (ICD-10) (2006) Mental and behaviour diseases, chapter 5: F00-F99

  • Jorde LB, Hasstedt SJ, Ritvo ER, Mason-Brothers A, Freeman BJ, Pingree C, McMahon WM, Petersen B, Jenson WR, Mo A (1991) Complex segregation analysis of autism. Am J Hum Genet 49(5):932–938

    PubMed  CAS  Google Scholar 

  • Kana RK, Murdaugh DL, Libero LE, Pennick MR, Wadsworth HM, Deshpande R et al. (2011) Probing the brain in autism using fMRI and diffusion tensor imaging. J Vis Exp 55:e3178

    Google Scholar 

  • Kar BR, Vijay N, Mishra S (2013) Development of cognitive and affective control networks and decision making. Prog Brain Res 202:347–368

    Article  PubMed  Google Scholar 

  • Kovas Y, Plomin R (2006) Generalist genes: implications for the cognitive sciences. Trends Cogn Sci 10(5): 198–203

    Article  PubMed  Google Scholar 

  • Kozima C, Yasuda Y (2007) Children-robot interaction: a pilot study in autism therapy. Prog Brain Res 164:385–400

    Article  PubMed  Google Scholar 

  • Lai CSL, Gerrelli D, Monaco AP, Fisher SE, Copp AJ (2003) FOXP2 expression during brain development coincides with adult sites of pathology in a severe speech and language disorder. Brain 126(11): 2455–2462

    Article  PubMed  Google Scholar 

  • Leslie AM (1994) Pretending and believing: issues in the theory of TOMM. Cognition 50:211–238

    Article  PubMed  CAS  Google Scholar 

  • Lord C, Rutter M, Couteur AL (1994) Autism diagnostic interview-revised: a revised version of a diagnostic interview for care-givers of individuals with possible pervasive developmental disorders. J Autism Dev Disord 24:659–685

    Article  PubMed  CAS  Google Scholar 

  • Main M (1996) Introduction to the special section on attachment and psychopathology: 2. Overview of the field of attachment. J Consult Clin Psychol 64:237–243

    Article  PubMed  CAS  Google Scholar 

  • Mareschal D, Johnson MH, Sirois S, Spratling MW, Thomas MSC, Westermann G (2007) Neuroconstructivism. How the brain constructs Cognition, vol 1. Oxford University Press, Oxford

  • Messinger DS (2005) A measure of early joy? In: Ekman P, Rosenberg EK (eds) What the face reveals: basic and applied studies of spontaneous expression union the Facial Action Coding System (FACS). Oxford University Press, Oxford, pp 350–353

    Google Scholar 

  • Michaud F, Caron S (2002) Roball, the rolling robot. Auton Robots 12:211–222

    Article  Google Scholar 

  • Michaud F, Salter T, Duquette A, Laplante JF (2007) Perspectives on mobile robots used as tools for pediatric rehabilitation. Assist Technol Spec Issue Intell Syst Pediatr Rehabil 19:14–19

    Google Scholar 

  • Misès R, Quemada N, Botbol M, Bursztejn C, Durand B, Garrabe J, et al. (2000) French classification of mental diseases in children and teenagers: CFTMEA R-2000

  • Nacewicz BM, Dalton KM, Johnstone T, Long M, McAuliff EM, Oakes TR, Alexander AL, Davidson RJ (2006) Amygdala volume and nonverbal social impairment in adolescent and adult males with autism. Arch Gen Psychiatry 63:1417–1428.

    Article  PubMed  Google Scholar 

  • Nadel J, Revel A, Andry P, Gaussier P (2004) Toward communication: first imitations in infants, low-functioning children with autism and robots. Interact Stud Soc Behav Commun Biol Artif Syst 5:45–54

    Article  Google Scholar 

  • Nishitani N, Hari R (2000) Temporal dynamics of cortical representation for action. Proc Natl Acad Sci USA 97:913–918

    Article  PubMed  CAS  Google Scholar 

  • Pellegrini AD (2009) The role of play in human development. Chapter 2: play: what is It? Oxford University Press, Oxford, pp 6–20

    Book  Google Scholar 

  • Pelphrey KA, Caster EJ (2008) Charting the typical and atypical development of the social brain. Dev Psychopathol 20:1081–1102

    Article  PubMed  Google Scholar 

  • Perry D, Hendler T, Shamay-Tsoory SG (2011) Can we share the joy of others? Empathic neural responses to distress vs. joy. Soc Cogn Affect Neurosci 7(8):909–916

    Article  PubMed  Google Scholar 

  • Pinker S (1994) The language instinct. William Morrow & Co., NY

    Google Scholar 

  • Plunkett K, Karmilloff-Smith A, Bates E, Elman JL, Johnson MH (1997) Connectionism and developmental psychology. J Child Psychol Psychiatry 38(1):53–80

    Article  PubMed  CAS  Google Scholar 

  • Pulvermüller F (1999) Words in the brain’s language. Behav Brain Sci 22:253–336

    Article  PubMed  Google Scholar 

  • Pulvermüller F (2003) The neuroscience of language: on brain circuits of words and serial order. Cambridge University Press, Cambridge

    Book  Google Scholar 

  • Quinn PC, Eimas PD (2000) The emergence of category representations during infancy: are separate perceptual and conceptual processes required? J Cognit Dev 1:55–61

    Article  Google Scholar 

  • Reed T, Peterson C (1990) A comparative study of autistic subjects’ performance at two levels of visual and cognitive perspective taking. J Aut Dev Disord 20:555–567

    Article  CAS  Google Scholar 

  • Robins DL, Fein D, Barton ML, Green JA (2001) The modified checklist for autism in toddlers: an initial study investigating the early detection of autism and pervasive developmental disorders. J Aut Dev Disord 31:131–144

    Article  CAS  Google Scholar 

  • Rolls ET (2004) The functions of the orbitofrontal cortex. Brain Cogn 55:11–29

    Article  PubMed  Google Scholar 

  • Russell JA (1989) Culture, scripts, and children’s understanding of emotion. In: Saarni C, Harris PL (eds) Children’s understanding of emotion. Cambridge University Press, Cambridge, pp 293–313

    Google Scholar 

  • Sagot I (2007) (sous la direction de Giannopulu I) Le ressenti émotionnel positif chez l’enfant à développement typique et atypique de 7 à 10 ans. Mémoire de fin d’études. UCP-EPP

  • Scassellati B, Admoni H (2012) Mataric M (2012) Robots for use in autism research. Annu Rev Biomed Eng 14:275–294

    Article  PubMed  CAS  Google Scholar 

  • Schopler E, Reichler RJ, DeVellis RG, Daly K (1980) Toward objective classification of childhood autism: childhood Autism Rating Scale (CARS). J Autism Dev Disord 10:91–103

    Article  PubMed  CAS  Google Scholar 

  • Siegel DJ (1999) The developing mind: how the relationships and the brain interact to shape who we are. Guilford Press, New York

    Google Scholar 

  • Smith PK (2005) Play: types and functions in human development. In: Ellis BJ, Bjorklund DF (eds) Origins of the social mind: evolutionary psychology and child development. Guilford Press, New York

    Google Scholar 

  • Sowell ER, Thompson PM, Leonard CM, Welcome SE, Toga AW (2004) Longitudinal mapping of cortical thickness and brain growth in normal children. J Neurosci 24:8223–8231

    Article  PubMed  CAS  Google Scholar 

  • Sykes NH, Toma C, Wilson N, Volpi EV, Sousa I, Pagnamenta AT, Tancredi R, Battaglia A, Maestrini E, Bailey AJ, Monaco AP (2009) Copy number variation and association analysis of SHANK3 as a candidate gene for autism in the IMGSAC collection. Eur J Hum Genet 17(10):1347–1353

    Article  PubMed  Google Scholar 

  • Szatmari P, Paterson AD, Zwaigenbaum L, Roberts W et al (2007) Mapping autism risk loci using genetic linkage and chromosomal rearrangements. Nat Genet 39:319–328

    Article  PubMed  CAS  Google Scholar 

  • Taylor BA, Hoch H, Potter B, Rodriguez A, Spinnato D, Kalaigian M (2005) Manipulating establishing operations to promote initiations toward peers in children with autism. Res Dev Disabilit 26:385–392

    Article  Google Scholar 

  • Thelen E, Smith LB (2006) Dynamic systems theories. In: Damon W, Lerner RM, Deanna Kuhn D, Siegler RS (eds) Handbook of child psychology, chapter 6. Wiley, New York, pp 258–312

    Google Scholar 

  • Tomasello M (1995) Language is not an instinct. Cognit Dev 10(1):131–156

    Article  Google Scholar 

  • Wainer J, Dautenhahn K, Robins B, Amirabdollahian F (2010) Collaborating with Kaspar: using an autonomous humanoid robot to foster cooperative dyadic play among children with autism. In: 10th IEEE-RAS international conference on humanoid robots (humanoids 10) Dec 2010, Nashville, TN

  • Williams E, Reddy V, Costal A (2001) Taking a closer look at functional play in children with autism. J Autism Dev Disord 31:67–77

    Article  PubMed  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Cognitive Neuroscience, Medical Engineering and Rehabilitation, Pierre and Marie Curie University, 4, Place Jussieu, 75005, Paris, France

    Irini Giannopulu

Authors
  1. Irini Giannopulu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Irini Giannopulu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Giannopulu, I. Multimodal interactions in typically and atypically developing children: natural versus artificial environments. Cogn Process 14, 323–331 (2013). https://doi.org/10.1007/s10339-013-0566-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10339-013-0566-0

Keywords

Search

Navigation