Animal Cognition

, Volume 21, Issue 5, pp 619–629 | Cite as

The function of primate multimodal communication

  • Marlen FröhlichEmail author
  • Carel P. van Schaik


Language is commonly narrowed down to speech, but human face-to-face communication is in fact an intrinsically multimodal phenomenon. Despite growing evidence that the communication of non-human primates, our main model for the evolution of language, is also inherently multimodal, most studies on primate communication have focused on either gestures or vocalizations in isolation. Accordingly, the biological function of multimodal signalling remains poorly understood. In this paper, we aim to merge the perspectives of comparative psychology and behavioural ecology on multimodal communication, and review existing studies in great apes for evidence of multimodal signal function based on content-based, efficacy-based and inter-signal interaction hypotheses. We suggest that cross-species comparisons of great ape interactions in both captive and wild settings will allow us to test the conditions in which these hypotheses apply. We expect such studies to provide novel insights into the function of speech-accompanying signals and cues, such as gestures, facial expressions, and eye gaze.


Communication Multimodality Socioecology Redundancy Refinement Complementarity Primates 



We thank Wouter Halfwerk, Maria van Noordwijk, David Leavens and Thibaud Gruber for fruitful discussions, and Kim Bard and two anonymous reviewers for useful comments on the manuscript.


M.F. was funded by the Sponsorship Society of the German Primate Centre, the A.H. Schultz Foundation and the Forschungskredit of the University of Zurich, Grant no. FK-17-106.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.


  1. Arbib MA, Liebal K, Pika S (2008) Primate vocalization, gesture, and the evolution of human language. Curr Anthropol 49:1053–1063PubMedCrossRefGoogle Scholar
  2. Argyle M (1988) Bodily Communication. Methuen & Co, New YorkGoogle Scholar
  3. Arnold K, Zuberbühler K (2006) Semantic combinations in primate calls. Nature 441:303PubMedCrossRefGoogle Scholar
  4. Bard KA, Leavens DA (2009) Socio-emotional factors in the development of joint attention in human and ape infants. In: Röska-Hardy LS, Neumann-Held EM (eds) Learning from animals? Examining the nature of human uniqueness. Psychology Press, Hove, pp 89–104Google Scholar
  5. Bard KA, Leavens DA (2014) The importance of development for comparative primatology. Annu Rev Anthropol 43:183–200CrossRefGoogle Scholar
  6. Bard KA, Dunbar S, Maguire-Herring V, Veira Y, Hayes KG, McDonald K (2014) Gestures and social-emotional communicative development in chimpanzee infants. Am J Primatol 76:14–29PubMedCrossRefGoogle Scholar
  7. Barrett L (2011) Beyond the brain: how body and environment shape animal and human minds. Princeton University Press, PrincetonGoogle Scholar
  8. Bateson P (2017) Behaviour, development and evolution. Open Book Publishers, CambridgeCrossRefGoogle Scholar
  9. Boesch C (2007) What makes us human (Homo sapiens)? The challenge of cognitive cross-species comparison. J Comp Psychol 121:227–240PubMedCrossRefGoogle Scholar
  10. Boesch C, Hohmann G, Marchant LF (2002) Behavioral diversity in chimpanzees and bonobos. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  11. Bradbury JW, Vehrencamp SL (1998) Principles of animal communication. Sinauer, SunderlandGoogle Scholar
  12. Bretherton I, Bates E (1979) The emergence of intentional communication. New Dir Child Adolesc Dev 1979:81–100CrossRefGoogle Scholar
  13. Brumm H, Slabbekoorn H (2005) Acoustic communication in noise. Advances in the study of behavior, vol 35. Academic Press, London, pp 151–209. CrossRefGoogle Scholar
  14. Byrne RW, Cartmill E, Genty E, Graham KE, Hobaiter C, Tanner J (2017) Great ape gestures: intentional communication with a rich set of innate signals. Anim Cogn 20:755–769PubMedPubMedCentralCrossRefGoogle Scholar
  15. Call J, Tomasello M (2007) The gestural communication of apes and monkeys. Lawrence Erlbaum Associates, New YorkGoogle Scholar
  16. Cartmill EA, Byrne RW (2007) Orangutans modify their gestural signaling according to their audience’s comprehension. Curr Biol 17:1345–1348PubMedCrossRefGoogle Scholar
  17. Clay Z, Zuberbühler K (2011) Bonobos extract meaning from call sequences. PLoS One 6:e18786PubMedPubMedCentralCrossRefGoogle Scholar
  18. Cowley S (2011) Distributed language. John Benjamins, AmsterdamCrossRefGoogle Scholar
  19. Ekman P, Friesen W (1969) The repertoire of non-verbal behavior: categories, origins, usage and coding. Semiotica 1:49–98CrossRefGoogle Scholar
  20. Fröhlich M (2017a) Catarrhine communication. In: Vonk J, Shackelford TK (eds) Encyclopedia of animal cognition and behavior. Springer, ChamGoogle Scholar
  21. Fröhlich M (2017b) Taking turns across channels: conversation-analytic tools in animal communication. Neurosci Biobehav Rev 80:201–209PubMedCrossRefGoogle Scholar
  22. Fröhlich M, Wittig RM, Pika S (2016) Should I stay or should I go? Initiation of joint travel in mother–infant dyads of two chimpanzee communities in the wild. Anim Cogn 19:483–500PubMedPubMedCentralCrossRefGoogle Scholar
  23. Fröhlich M, Müller G, Zeiträg C, Wittig RM, Pika S (2017) Gestural development of chimpanzees in the wild: the impact of interactional experience. Anim Behav 134:271CrossRefGoogle Scholar
  24. Genty E, Clay Z, Hobaiter C, Zuberbühler K (2014) Multi-modal use of a socially directed call in Bonobos. PLoS One 9:e84738PubMedPubMedCentralCrossRefGoogle Scholar
  25. Ghazanfar AA (2013) Multisensory vocal communication in primates and the evolution of rhythmic speech. Behav Ecol Sociobiol 67:1441–1448CrossRefGoogle Scholar
  26. Ghazanfar AA, Rendall D (2008) Evolution of human vocal production vol 18, vol 11. Cell Press, CambridgeGoogle Scholar
  27. Gillespie-Lynch K, Greenfield PM, Lyn H, Savage-Rumbaugh S (2014) Gestural and symbolic development among apes and humans: support for a multimodal theory of language evolution. Front Psychol 5:1228PubMedPubMedCentralCrossRefGoogle Scholar
  28. Goldin-Meadow S (1999) The role of gesture in communication and thinking. Trends Cogn Sci 3:419–429PubMedCrossRefGoogle Scholar
  29. Goldin-Meadow S, Wagner SM (2005) How our hands help us learn. Trends Cogn Sci 9:234–241PubMedCrossRefGoogle Scholar
  30. Gould SJ, Lewontin RC (1979) The spandrels of San Marco and the Panglossian paradigm: a critique of the adaptationist programme. Proc R Soc Lond B 205:581–598PubMedCrossRefGoogle Scholar
  31. Graham JA, Argyle M (1975) A cross-cultural study of the communication of extra-verbal meaning by gestures. Int J Psychol 10:57–67CrossRefGoogle Scholar
  32. Graham KE, Furuichi T, Byrne RW (2016) The gestural repertoire of the wild bonobo (Pan paniscus): a mutually understood communication system. Anim Cogn 20:1–7Google Scholar
  33. Guilford T, Dawkins MS (1991) Receiver psychology and the evolution of animal signals. Anim Behav 42:1–14CrossRefGoogle Scholar
  34. Halfwerk W, Slabbekoorn H (2015) Pollution going multimodal: the complex impact of the human-altered sensory environment on animal perception and performance. Biol Lett 11:20141051PubMedPubMedCentralCrossRefGoogle Scholar
  35. Halfwerk W, Page Rachel A, Taylor Ryan C, Wilson Preston S, Ryan Michael J (2014) Crossmodal comparisons of signal components allow for relative-distance assessment. Curr Biol 24:1751–1755PubMedCrossRefGoogle Scholar
  36. Hauser MD, Teixidor P, Field L, Flaherty R (1993) Food-elicited calls in chimpanzees: effects of food quantity and divisibility? Anim Behav 45:817–819CrossRefGoogle Scholar
  37. Hebets EA, Papaj DR (2005) Complex signal function: developing a framework of testable hypotheses. Behav Ecol Sociobiol 57:197–214CrossRefGoogle Scholar
  38. Hebets EA, Vink CJ (2007) Experience leads to preference: experienced females prefer brush-legged males in a population of syntopic wolf spiders. Behav Ecol 18:1010–1020CrossRefGoogle Scholar
  39. Higham JP, Hebets EA (2013) An introduction to multimodal communication. Behav Ecol Sociobiol 67:1381–1388CrossRefGoogle Scholar
  40. Hobaiter C, Byrne RW (2011) Serial gesturing by wild chimpanzees: its nature and function for communication. Anim Cogn 14:827–838PubMedCrossRefGoogle Scholar
  41. Hobaiter C, Byrne RW, Zuberbühler K (2017) Wild chimpanzees’ use of single and combined vocal and gestural signals. Behav Ecol Sociobiol 71:96PubMedPubMedCentralCrossRefGoogle Scholar
  42. Holle H, Gunter TC (2007) The role of iconic gestures in speech disambiguation: ERP evidence. J Cogn Neurosci 19:1175–1192PubMedCrossRefGoogle Scholar
  43. Hopkins WD, Taglialatela JD, Leavens D (2007) Chimpanzees differentially produce novel vocalizations to capture the attention of a human. Anim Behav 73:281–286PubMedPubMedCentralCrossRefGoogle Scholar
  44. Jawor JM, Gray N, Beall SM, Breitwisch R (2004) Multiple ornaments correlate with aspects of condition and behaviour in female northern cardinals, Cardinalis cardinalis. Anim Behav 67:875–882CrossRefGoogle Scholar
  45. Johnstone RA (1996) Multiple displays in animal communication: ‘backup signals’ and ‘multiple messages’. Philos Trans R Soc Lond B Biol Sci 351:329–338CrossRefGoogle Scholar
  46. Katsu N, Yamada K, Nakamichi M (2017) Influence of social interactions with nonmother females on the development of call usage in Japanese macaques. Anim Behav 123:267–276CrossRefGoogle Scholar
  47. Kelly SD, Barr DJ, Church R, Lynch K (1999) Offering a hand to pragmatic understanding: the role of speech and gesture in comprehension and memory. J Mem Lang 40:577–592CrossRefGoogle Scholar
  48. Kendon A (2004) Gesture: visible action as utterance. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  49. King BJ, Shanker SG (2003) How can we know the dancer from the dance: the dynamic nature of African great ape social communication. Anthropol Theory 3:5–23CrossRefGoogle Scholar
  50. Laporte MNC, Zuberbühler K (2011) The development of a greeting signal in wild chimpanzees. Dev Sci 14:1220–1234PubMedCrossRefGoogle Scholar
  51. Leavens DA, Bard KA (2011) Environmental influences on joint attention in great apes: implications for human cognition. J Cogn Educ Psychol 10:9–31CrossRefGoogle Scholar
  52. Leavens DA, Hopkins WD (2005) Multimodal concomitants of manual gesture by chimpanzees: influence of food size and distance. Gesture 5:73–88CrossRefGoogle Scholar
  53. Leavens DA, Russell JL, Hopkins WD (2005) Intentionality as measured in the persistence and elaboration of communication by chimpanzees (Pan troglodytes). Child Dev 76:291–306PubMedPubMedCentralCrossRefGoogle Scholar
  54. Leavens DA, Russell J, Hopkins W (2010) Multimodal communication by captive chimpanzees (Pan troglodytes). Anim Cogn 13:33–40PubMedCrossRefGoogle Scholar
  55. Levinson SC (2016) Turn-taking in human communication–origins and implications for language processing. Trends Cogn Sci 20:6–14PubMedCrossRefGoogle Scholar
  56. Levinson SC, Holler J (2014) The origin of human multi-modal communication. Philos Trans R Soc Lond B Biol Sci 369:20130302PubMedPubMedCentralCrossRefGoogle Scholar
  57. Liebal K, Pika S, Tomasello M (2006) Gestural communication of orangutans (Pongo pygmaeus). Gesture 6:1–38CrossRefGoogle Scholar
  58. Liebal K, Waller BM, Burrows AM, Slocombe KE (2014) Primate communication: a multimodal approach. Cambridge University Press, CambridgeGoogle Scholar
  59. Macaluso E, Frith CD, Driver J (2000) Modulation of human visual cortex by crossmodal spatial attention. Science 289:1206–1208PubMedCrossRefGoogle Scholar
  60. Marler P (1961) The logical analysis of animal communication. J Theor Biol 1:295–317PubMedCrossRefGoogle Scholar
  61. Massaro DW (1998) Perceiving talking faces: from speech perception to a behavioral principle. MIT Press, CambridgeGoogle Scholar
  62. McGurk H, MacDonald J (1976) Hearing lips and seeing voices. Nature 264:746–748PubMedCrossRefGoogle Scholar
  63. McNeill D (1992) Hand and mind: what gestures reveal about thought. University of Chicago Press, ChicagoGoogle Scholar
  64. Micheletta J, Engelhardt A, Matthews LEE, Agil M, Waller BM (2013) Multicomponent and multimodal lipsmacking in crested macaques (Macaca nigra). Am J Primatol 75:763–773PubMedCrossRefGoogle Scholar
  65. Michelsen A, Andersen BB, Storm J, Kirchner WH, Lindauer M (1992) How honeybees perceive communication dances, studied by means of a mechanical model. Behav Ecol Sociobiol 30:143–150CrossRefGoogle Scholar
  66. Møller A, Pomiankowski A (1993) Why have birds got multiple sexual ornaments? Behav Ecol Sociobiol 32:167–176Google Scholar
  67. Munoz NE, Blumstein DT (2012) Multisensory perception in uncertain environments. Behav Ecol 23:457–462CrossRefGoogle Scholar
  68. Partan SR (2002) Single and multichannel signal composition: facial expressions and vocalizations of rhesus macaques (Macaca mulatta). Behaviour 139:993–1027CrossRefGoogle Scholar
  69. Partan SR (2013) Ten unanswered questions in multimodal communication. Behav Ecol Sociobiol 67:1523–1539PubMedPubMedCentralCrossRefGoogle Scholar
  70. Partan SR (2016) Multimodal shifts in noise: switching channels to communicate through rapid environmental change. Anim Behav 124:325CrossRefGoogle Scholar
  71. Partan SR, Marler P (1999) Communication goes multimodal. Science 283:1272–1273PubMedCrossRefGoogle Scholar
  72. Partan SR, Marler P (2005) Issues in the classification of multimodal communication signals. Am Nat 166:231–245PubMedCrossRefGoogle Scholar
  73. Pika S, Fröhlich M (2018) Gestural acquisition in great apes: the social negotiation hypothesis. Anim Cogn 1:15Google Scholar
  74. Pika S, Liebal K, Tomasello M (2003) Gestural communication in young gorillas (Gorilla gorilla): gestural repertoire, learning, and use. Am J Primatol 60:95–111PubMedCrossRefGoogle Scholar
  75. Pika S, Liebal K, Tomasello M (2005) Gestural communication in subadult bonobos (Pan paniscus): repertoire and use. Am J Primatol 65:39–61PubMedCrossRefGoogle Scholar
  76. Pollick AS, de Waal FBM (2007) Ape gestures and language evolution. Proc Natl Acad Sci 104:8184–8189PubMedCrossRefGoogle Scholar
  77. Rowe C (1999a) One signal or two? Science 284:741 hCrossRefGoogle Scholar
  78. Rowe C (1999b) Receiver psychology and the evolution of multicomponent signals. Anim Behav 58:921–931PubMedCrossRefGoogle Scholar
  79. Ruxton G, Schaefer H (2011) Resolving current disagreements and ambiguities in the terminology of animal communication. J Evol Biol 24:2574–2585PubMedCrossRefGoogle Scholar
  80. Rybak F, Sureau G, Aubin T (2002) Functional coupling of acoustic and chemical signals in the courtship behaviour of the male Drosophila melanogaster. Proc R Soc Lond B Biol Sci 269:695–701CrossRefGoogle Scholar
  81. Scott-Phillips TC, Dickins TE, West SA (2011) Evolutionary theory and the ultimate–proximate distinction in the human behavioral sciences. Perspect Psychol Sci 6:38–47PubMedCrossRefGoogle Scholar
  82. Shanker SG, King BJ (2002) The emergence of a new paradigm in ape language research: beyond interactionism. Behav Brain Sci 25:646–651Google Scholar
  83. Slocombe KE, Waller BM, Liebal K (2011) The language void: the need for multimodality in primate communication research. Anim Behav 81:919–924CrossRefGoogle Scholar
  84. Smith WJ (1965) Message, meaning, and context in ethology. Am Nat 99:405–409CrossRefGoogle Scholar
  85. Snowdon CT, Hausberger M (eds) (1997) Social influences on vocal development. Cambridge University Press, CambridgeGoogle Scholar
  86. Soto-Faraco S, Calabresi M, Navarra J, Werker J, Lewkowicz DJ (2012) The development of audiovisual speech perception. Multisensory Development. Oxford University Press, Oxford, pp 207–228Google Scholar
  87. Spence C, Nicholls ME, Gillespie N, Driver J (1998) Cross-modal links in exogenous covert spatial orienting between touch, audition, and vision. Percept Psychophys 60:544–557PubMedCrossRefGoogle Scholar
  88. Sumby WH, Pollack I (1954) Visual contribution to speech intelligibility in noise. J Acoust Soc Am 26:212–215CrossRefGoogle Scholar
  89. Taborsky B, Oliveira RF (2012) Social competence: an evolutionary approach. Trends Ecol Evol 27:679–688PubMedCrossRefGoogle Scholar
  90. Taglialatela JP, Russell JL, Schaeffer JA, Hopkins WD (2011) Chimpanzee vocal signaling points to a multimodal origin of human language. PLoS One 6:e18852PubMedPubMedCentralCrossRefGoogle Scholar
  91. Taglialatela JP et al (2015) Multimodal communication in chimpanzees. Am J Primatol 77:1143–1148PubMedPubMedCentralCrossRefGoogle Scholar
  92. Tinbergen N (1963) On aims and methods of ethology. Ethology 20:410–433Google Scholar
  93. Tomasello M (2008) Origins of human communication, vol 2008. MIT Press, CambridgeGoogle Scholar
  94. Tomasello M, Call J, Nagell K, Olguin R, Carpenter M (1994) The learning and use of gestural signals by young chimpanzees: a trans-generational study. Primates 35:137–154CrossRefGoogle Scholar
  95. Tomasello M, Call J, Warren J, Frost T, Carpenter M, Nagell K (1997) The ontogeny of chimpanzee gestural signals: a comparison across groups and generations. Evol Commun 1:224–259Google Scholar
  96. Townsend SW et al (2016) Exorcising Grice’s ghost: an empirical approach to studying intentional communication in animals. Biol Rev 92:1427–1433PubMedCrossRefGoogle Scholar
  97. van Schaik CP (2004) Among orangutans: red apes and the rise of human culture. Belknap, CambridgeGoogle Scholar
  98. Van Schaik CP (2016) The primate origins of human nature. Wiley, OxfordGoogle Scholar
  99. Vigliocco G, Perniss P, Vinson D (2014) Language as a multimodal phenomenon: implications for language learning, processing and evolution. The Royal Society, LondonGoogle Scholar
  100. Waller BM, Liebal K, Burrows AM, Slocombe KE (2013) How can a multimodal approach to primate communication help us understand the evolution of communication? Evol Psychol 11:147470491301100305CrossRefGoogle Scholar
  101. Wedekind C, Meyer P, Frischknecht M, Niggli UA, Pfander H (1998) Different carotenoids and potential information content of red coloration of male three-spined stickleback. J Chem Ecol 24:787–801CrossRefGoogle Scholar
  102. Wheeler BC, Fischer J (2012) Functionally referential signals: a promising paradigm whose time has passed. Evol Anthropol Issues News Rev 21:195–205CrossRefGoogle Scholar
  103. Wilke C, Kavanagh E, Donnellan E, Waller BM, Machanda ZP, Slocombe KE (2017) Production of and responses to unimodal and multimodal signals in wild chimpanzees, Pan troglodytes schweinfurthii. Anim Behav 123:305–316CrossRefGoogle Scholar
  104. Wilson EO (1976) Sociobiology. Belknap, CambridgeGoogle Scholar
  105. Zuk M, Ligon JD, Thornhill R (1992) Effects of experimental manipulation of male secondary sex characters on female mate preference in red jungle fowl. Anim Behav 44:999–1006CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of AnthropologyUniversity of ZurichZurichSwitzerland

Personalised recommendations