The Default Mode of Primate Vocal Communication and Its Neural Correlates

  • Asif A. GhazanfarEmail author


Communication is, by default, a multisensory phenomenon. In support of this contention, I review evidence that beyond the familiar ideas about audiovisual speech in humans, there is also automatic integration of faces and voices during vocal perception by monkeys and apes. At the neural level, this integration is mediated, in part, by interactions between 'unimodal' sensory areas and association areas in the temporal lobe. How these neural interactions develop may be driven by species-typical social experiences. The overwhelming evidence from the studies reviewed here, and numerous other studies from different domains of neuroscience, all converge on the idea that, like the behavior of communication itself, the neocortex is fundamentally multisensory. It is not confined to a few ‘sensu comune’ in the association cortices. This does not mean, however, that the neocortex is uniformly multisensory, but rather that cortical areas maybe weighted differently by ‘extra’-modal inputs depending on the task at hand and its context.


Speech Perception Auditory Cortex Vocal Tract Human Infant Local Field Potential 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The author gratefully acknowledges the scientific contributions and numerous discussions with the following people: Chand Chandrasekaran, Kari Hoffman, David Lewkowicz, Joost Maier, and Hjalmar Turesson. This work was supported by NIH R01NS054898 and NSF BCS-0547760 CAREER Award.


  1. Adachi I, Kuwahata H, Fujita K, Tomonaga M, Matsuzawa T (2006) Japanese macaques form a cross-modal representation of their own species in their first year of life. Primates 47: 350–354PubMedCrossRefGoogle Scholar
  2. Antinucci F (1989) Systematic comparison of early sensorimotor development. In: Antinucci F (ed) Cognitive structure and development in nonhuman primates.: Lawrence Erlbaum Associates, Hillsdale, NJ, pp 67–85Google Scholar
  3. Barnes CL, Pandya DN (1992) Efferent cortical connections of multimodal cortex of the superior temporal sulcus in the rhesus-monkey. J Comp Neurol 318:222–244PubMedCrossRefGoogle Scholar
  4. Barraclough NE, Xiao D, Baker CI, Oram MW, Perrett DI (2005) Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. J Cogn Neurosci 17:377–391PubMedCrossRefGoogle Scholar
  5. Benevento LA, Fallon J, Davis BJ, Rezak M (1977) Auditory-visual interactions in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey. Exp Neurol 57:849–872PubMedCrossRefGoogle Scholar
  6. Bernstein LE, Auer ET, Takayanagi S (2004) Auditory speech detection in noise enhanced by lipreading. Speech Commun 44:5–18CrossRefGoogle Scholar
  7. Besle J, Fort A, Delpuech C, Giard MH (2004) Bimodal speech: early suppressive visual effects in human auditory cortex. Eur J Neurosci 20:2225–2234PubMedCrossRefGoogle Scholar
  8. Bizley JK, Nodal FR, Bajo VM, Nelken I, King AJ (2007) Physiological and anatomical evidence for multisensory interactions in auditory cortex. Cereb Cortex 17:2172–2189PubMedCrossRefGoogle Scholar
  9. Bruce C, Desimone R, Gross CG (1981) Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. J Neurophysiol 46:369–384PubMedGoogle Scholar
  10. Cappe C, Barone P (2005) Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey. Eur J Neurosci 22:2886–2902PubMedCrossRefGoogle Scholar
  11. Chandrasekaran C, Ghazanfar AA (2009) Different neural frequency bands integrate faces and voices differently in the superior temporal sulcus. J Neurophysiol 101:773–788PubMedCrossRefGoogle Scholar
  12. Cheney DL, Seyfarth RM (1982) How vervet monkeys perceive their grunts – field playback experiments. Animal Behav 30:739–751CrossRefGoogle Scholar
  13. de la Mothe LA, Blumell S, Kajikawa Y, Hackett TA (2006) Cortical connections of the auditory cortex in marmoset monkeys: Core and medial belt regions. J Comp Neurol 496:27–71PubMedCrossRefGoogle Scholar
  14. Driver J, Noesselt T (2008) Multisensory interplay reveals crossmodal influences on ‘sensory-specific’ brain regions, neural responses, and judgments. Neuron 57:11–23PubMedCrossRefGoogle Scholar
  15. Ettlinger G, Wilson WA (1990) Cross-modal performance: behavioural processes, phylogenetic considerations and neural mechanisms. Behav Brain Res 40:169–192PubMedCrossRefGoogle Scholar
  16. Evans TA, Howell S, Westergaard GC (2005) Auditory-visual cross-modal perception of communicative stimuli in tufted capuchin monkeys (Cebus apella). J Exp Psychol-Anim Behav Proc 31:399–406CrossRefGoogle Scholar
  17. Fitch WT (1997) Vocal tract length and formant frequency dispersion correlate with body size in rhesus macaques. J Acoust Soc Am 102:1213–1222PubMedCrossRefGoogle Scholar
  18. Fitch WT, Hauser MD (1995) Vocal production in nonhuman-primates - acoustics, physiology, and functional constraints on honest advertisement. Am J Primatol 37:191–219CrossRefGoogle Scholar
  19. Fu KMG, Johnston TA, Shah AS, Arnold L, Smiley J, Hackett TA, Garraghty PE, Schroeder CE (2003) Auditory cortical neurons respond to somatosensory stimulation. J Neurosci 23: 7510–7515PubMedGoogle Scholar
  20. Fu KMG, Shah AS, O’Connell MN, McGinnis T, Eckholdt H, Lakatos P, Smiley J, Schroeder CE (2004) Timing and laminar profile of eye-position effects on auditory responses in primate auditory cortex. J Neurophysiol 92:3522–3531PubMedCrossRefGoogle Scholar
  21. Ghazanfar AA, Logothetis NK (2003) Facial expressions linked to monkey calls. Nature 423:937–938PubMedCrossRefGoogle Scholar
  22. Ghazanfar AA, Schroeder CE (2006) Is neocortex essentially multisensory? Trends Cogn Sci 10:278–285PubMedCrossRefGoogle Scholar
  23. Ghazanfar AA, Rendall D (2008) Evolution of human vocal production. Curr Biol 18:R457–R460PubMedCrossRefGoogle Scholar
  24. Ghazanfar AA, Nielsen K, Logothetis NK (2006) Eye movements of monkeys viewing vocalizing conspecifics. Cognition 101:515–529PubMedCrossRefGoogle Scholar
  25. Ghazanfar AA, Chandrasekaran C, Logothetis NK (2008) Interactions between the superior temporal sulcus and auditory cortex mediate dynamic face/voice integration in rhesus monkeys. J Neurosci 28:4457–4469PubMedCrossRefGoogle Scholar
  26. Ghazanfar AA, Maier JX, Hoffman KL, Logothetis NK (2005) Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. J Neurosci 25:5004–5012PubMedCrossRefGoogle Scholar
  27. Ghazanfar AA, Turesson HK, Maier JX, van Dinther R, Patterson RD, Logothetis NK (2007) Vocal tract resonances as indexical cues in rhesus monkeys. Curr Biol 17:425–430PubMedCrossRefGoogle Scholar
  28. Gibson KR (1991) Myelination and behavioral development: A comparative perspective on questions of neoteny, altriciality and intelligence. In: Gibson KR, Petersen AC (eds) Brain maturation and cognitive development: comparative and cross-cultural perspective. Aldine de Gruyter, New York, pp 29–63Google Scholar
  29. Gogate LJ, Walker-Andrews AS, Bahrick LE (2001) The intersensory origins of word comprehension: an ecological-dynamic systems view. Develop Sci 4:1–18CrossRefGoogle Scholar
  30. Gothard KM, Battaglia FP, Erickson CA, Spitler KM, Amaral DG (2007) Neural responses to facial expression and face identity in the monkey amygdala. J Neurophysiol 97:1671–1683PubMedCrossRefGoogle Scholar
  31. Hackett TA, Stepniewska I, Kaas JH (1999) Prefrontal connections of the parabelt auditory cortex in macaque monkeys. Brain Res 817:45–58PubMedCrossRefGoogle Scholar
  32. Hackett TA, De La Mothe LA, Ulbert I, Karmos G, Smiley J, Schroeder CE (2007a) Multisensory convergence in auditory cortex, II. Thalamocortical connections of the caudal superior temporal plane. J Comp Neurol 502:924–952PubMedCrossRefGoogle Scholar
  33. Hackett TA, Smiley JF, Ulbert I, Karmos G, Lakatos P, de la Mothe LA, Schroeder CE (2007b) Sources of somatosensory input to the caudal belt areas of auditory cortex. Perception 36:1419–1430PubMedCrossRefGoogle Scholar
  34. Harries MH, Perrett DI (1991) Visual processing of faces in temporal cortex - physiological evidence for a modular organization and possible anatomical correlates. J Cogn Neurosci 3:9–24CrossRefGoogle Scholar
  35. Hauser MD, Ybarra MS (1994) The role of lip configuration in monkey vocalizations - experiments using xylocaine as a nerve block. Brain Lang 46:232–244PubMedCrossRefGoogle Scholar
  36. Hauser MD, Evans CS, Marler P (1993) The role of articulation in the production of rhesus-monkey, Macaca-Mulatta, vocalizations. Anim Behav 45:423–433CrossRefGoogle Scholar
  37. Ito T, Tiede M, Ostry DJ (2009) Somatosensory function in speech perception. Proc Natl Acad Sci U S A 106:1245–1248PubMedCrossRefGoogle Scholar
  38. Iyengar S, Qi H, Jain N, Kaas JH (2007) Cortical and thalamic connections of the representations of the teeth and tongue in somatosensory cortex of new world monkeys. J Comp Neurol 501:95–120PubMedCrossRefGoogle Scholar
  39. Izumi A, Kojima S (2004) Matching vocalizations to vocalizing faces in a chimpanzee (Pan troglodytes). Anim Cogn 7:179–184PubMedCrossRefGoogle Scholar
  40. Jiang JT, Alwan A, Keating PA, Auer ET, Bernstein LE (2002) On the relationship between face movements, tongue movements, and speech acoustics. Eurasip J Appl Sig Proc 2002: 1174–1188CrossRefGoogle Scholar
  41. Jordan KE, Brannon EM, Logothetis NK, Ghazanfar AA (2005) Monkeys match the number of voices they hear with the number of faces they see. Curr Biol 15:1034–1038PubMedCrossRefGoogle Scholar
  42. Kayser C, Logothetis NK (2009) Directed interactions between auditory and superior temporal cortices and their role in sensory integration. Front Integr Neurosci 3:7Google Scholar
  43. Kayser C, Petkov CI, Logothetis NK (2008) Visual modulation of neurons in auditory cortex. Cereb Cortex 18:1560–1574PubMedCrossRefGoogle Scholar
  44. Kayser C, Petkov CI, Augath M, Logothetis NK (2005) Integration of touch and sound in auditory cortex. Neuron 48:373–384PubMedCrossRefGoogle Scholar
  45. Kayser C, Petkov CI, Augath M, Logothetis NK (2007) Functional imaging reveals visual modulation of specific fields in auditory cortex. J Neurosci 27:1824–1835PubMedCrossRefGoogle Scholar
  46. Klin A, Jones W, Schultz R, Volkmar F, Cohen D (2002) Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Archiv Gen Psychiatry 59:809–816CrossRefGoogle Scholar
  47. Konner M (1991) Universals of behavioral development in relation to brain myelination. In: Gibson KR, Petersen AC (eds) Brain maturation and cognitive development: comparative and cross-cultural perspectives. Aldine de Gruyter, New York, pp 181–223Google Scholar
  48. Kuhl PK, Williams KA, Meltzoff AN (1991) Cross-modal speech perception in adults and infants using nonspeech auditory stimuli. J Exp Psychol: Human Percept Perform 17:829–840CrossRefGoogle Scholar
  49. Kuraoka K, Nakamura K (2007) Responses of single neurons in monkey amygdala to facial and vocal emotions. J Neurophysiol 97:1379–1387PubMedCrossRefGoogle Scholar
  50. Lakatos P, Chen C-M, O'Connell MN, Mills A, Schroeder CE (2007) Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 53:279–292PubMedCrossRefGoogle Scholar
  51. Lakatos P, Shah AS, Knuth KH, Ulbert I, Karmos G, Schroeder CE (2005) An oscillatory hierarchy controlling neuronal excitability and stimulus processing in the auditory cortex. J Neurophysiol 94:1904–1911PubMedCrossRefGoogle Scholar
  52. Lansing IR, McConkie GW (2003) Word identification and eye fixation locations in visual and visual-plus-auditory presentations of spoken sentences. Percept Psychophys 65:536–552PubMedCrossRefGoogle Scholar
  53. Lewkowicz DJ, Ghazanfar AA (2006) The decline of cross-species intersensory perception in human infants. Proc Natl Acad Sci U S A 103:6771–6774PubMedCrossRefGoogle Scholar
  54. Lewkowicz DJ, Sowinski R, Place S (2008) The decline of cross-species intersensory perception in human infants: underlying mechanisms and its developmental persistence. Brain Res 1242:291–302PubMedCrossRefGoogle Scholar
  55. Malkova L, Heuer E, Saunders RC (2006) Longitudinal magnetic resonance imaging study of rhesus monkey brain development. Eur J Neurosci 24:3204–3212PubMedCrossRefGoogle Scholar
  56. McGurk H, MacDonald J (1976) Hearing lips and seeing voices. Nature 264:229–239CrossRefGoogle Scholar
  57. Oram MW, Perrett DI (1994) Responses of anterior superior temporal polysensory (Stpa) neurons to biological motion stimuli. J Cogn Neurosci 6:99–116CrossRefGoogle Scholar
  58. Palombit RA, Cheney DL, Seyfarth RM (1999) Male grunts as mediators of social interaction with females in wild chacma baboons (Papio cynocephalus ursinus). Behaviour 136:221–242CrossRefGoogle Scholar
  59. Parr LA (2004) Perceptual biases for multimodal cues in chimpanzee (Pan troglodytes) affect recognition. Anim Cogn 7:171–178PubMedCrossRefGoogle Scholar
  60. Patterson ML, Werker JF (2003) Two-month-old infants match phonetic information in lips and voice. Develop Sci 6:191–196CrossRefGoogle Scholar
  61. Pevsner J (2002) Leonardo da Vinci’s contributions to neuroscience. Trends Neurosci 25:217–220PubMedCrossRefGoogle Scholar
  62. Robinson DA, Fuchs AF (1969) Eye movements evoked by stimulation of frontal eye fields. J Neurophysiol 32:637–648PubMedGoogle Scholar
  63. Romanski LM, Bates JF, Goldman-Rakic PS (1999) Auditory belt and parabelt projections to the prefrontal cortex in the rhesus monkey. J Comp Neurol 403:141–157PubMedCrossRefGoogle Scholar
  64. Romanski LM, Averbeck BB, Diltz M (2005) Neural representation of vocalizations in the primate ventrolateral prefrontal cortex. J Neurophysiol 93:734–747PubMedCrossRefGoogle Scholar
  65. Rosenblum LD (2005) Primacy of multimodal speech perception. In: Pisoni DB, Remez RE (eds) Handbook of speech perception. Blackwell, Malden, MA, pp 51–78CrossRefGoogle Scholar
  66. Rosenblum LD (2008) Speech perception as a multimodal phenomenon. Curr Direct Psychol Sci 17:405–409CrossRefGoogle Scholar
  67. Sacher GA, Staffeldt EF (1974) Relation of gestation time to brain weight for placental mammals: implications for the theory of vertebrate growth. Am Naturalist 108:593–615CrossRefGoogle Scholar
  68. Sams M, Mottonen R, Sihvonen T (2005) Seeing and hearing others and oneself talk. Cogn Brain Res 23:429–435CrossRefGoogle Scholar
  69. Schall JD, Morel A, King DJ, Bullier J (1995) Topography of visual cortex connections with frontal eye field in macaque: convergence and segregation of processing streams. J Neurosci 15: 4464–4487PubMedGoogle Scholar
  70. Schroeder CE, Foxe JJ (2002) The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Cogn Brain Res 14:187–198CrossRefGoogle Scholar
  71. Schroeder CE, Lindsley RW, Specht C, Marcovici A, Smiley JF, Javitt DC (2001) Somatosensory input to auditory association cortex in the macaque monkey. J Neurophysiol 85:1322–1327PubMedGoogle Scholar
  72. Schwartz J-L, Berthommier F, Savariaux C (2004) Seeing to hear better: evidence for early audio-visual interactions in speech identification. Cognition 93:B69–B78PubMedCrossRefGoogle Scholar
  73. Seltzer B, Pandya DN (1989) Frontal-lobe connections of the superior temporal sulcus in the rhesus-monkey. J Comp Neurol 281:97–113PubMedCrossRefGoogle Scholar
  74. Seltzer B, Pandya DN (1994) Parietal, temporal, and occipital projections to cortex of the superior temporal sulcus in the rhesus monkey: a retrograde tracer study. J Comp Neurol 343:445–463PubMedCrossRefGoogle Scholar
  75. Smiley JF, Hackett TA, Ulbert I, Karmas G, Lakatos P, Javitt DC, Schroeder CE (2007) Multisensory convergence in auditory cortex, I. Cortical connections of the caudal superior temporal plane in macaque monkeys. J Comp Neurol 502:894–923PubMedCrossRefGoogle Scholar
  76. Sugihara T, Diltz MD, Averbeck BB, Romanski LM (2006) Integration of auditory and visual communication information in the primate ventrolateral prefrontal cortex. J Neurosci 26: 11138–11147PubMedCrossRefGoogle Scholar
  77. van Wassenhove V, Grant KW, Poeppel D (2005) Visual speech speeds up the neural processing of auditory speech. Proc Natl Acad Sci US A 102:1181–1186CrossRefGoogle Scholar
  78. Vatikiotis-Bateson E, Eigsti IM, Yano S, Munhall KG (1998) Eye movement of perceivers during audiovisual speech perception. Percept Psychophys 60:926–940PubMedCrossRefGoogle Scholar
  79. Werner-Reiss U, Kelly KA, Trause AS, Underhill AM, Groh JM (2003) Eye position affects activity in primary auditory cortex of primates. Curr Biol 13:554–562PubMedCrossRefGoogle Scholar
  80. Yehia H, Rubin P, Vatikiotis-Bateson E (1998) Quantitative association of vocal-tract and facial behavior. Speech Commun 26:23–43CrossRefGoogle Scholar
  81. Yehia HC, Kuratate T, Vatikiotis-Bateson E (2002) Linking facial animation, head motion and speech acoustics. J Phonet 30:555–568CrossRefGoogle Scholar
  82. Zangehenpour S, Ghazanfar AA, Lewkowicz DJ, Zatorre RJ (2008) Heterochrony and cross-species intersensory matching by infant vervet monkeys. PLoS ONE 4:e4302CrossRefGoogle Scholar

Copyright information

© Springer Science + Business Media, LLC 2010

Authors and Affiliations

  1. 1.Departments of Psychology and Ecology & Evolutionary BiologyNeuroscience Institute, Princeton UniversityPrincetonUSA

Personalised recommendations