Skip to main content
SpringerLink
Log in

The function of primate multimodal communication

  • Review
  • Published:
Animal Cognition Aims and scope Submit manuscript

Abstract

Language is commonly narrowed down to speech, but human face-to-face communication is in fact an intrinsically multimodal phenomenon. Despite growing evidence that the communication of non-human primates, our main model for the evolution of language, is also inherently multimodal, most studies on primate communication have focused on either gestures or vocalizations in isolation. Accordingly, the biological function of multimodal signalling remains poorly understood. In this paper, we aim to merge the perspectives of comparative psychology and behavioural ecology on multimodal communication, and review existing studies in great apes for evidence of multimodal signal function based on content-based, efficacy-based and inter-signal interaction hypotheses. We suggest that cross-species comparisons of great ape interactions in both captive and wild settings will allow us to test the conditions in which these hypotheses apply. We expect such studies to provide novel insights into the function of speech-accompanying signals and cues, such as gestures, facial expressions, and eye gaze.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Modified from Fröhlich (2017b). (Color figure online)

Similar content being viewed by others

References

  • Arbib MA, Liebal K, Pika S (2008) Primate vocalization, gesture, and the evolution of human language. Curr Anthropol 49:1053–1063

    Article  PubMed  Google Scholar 

  • Argyle M (1988) Bodily Communication. Methuen & Co, New York

    Google Scholar 

  • Arnold K, Zuberbühler K (2006) Semantic combinations in primate calls. Nature 441:303

    Article  PubMed  CAS  Google Scholar 

  • Bard KA, Leavens DA (2009) Socio-emotional factors in the development of joint attention in human and ape infants. In: Röska-Hardy LS, Neumann-Held EM (eds) Learning from animals? Examining the nature of human uniqueness. Psychology Press, Hove, pp 89–104

    Google Scholar 

  • Bard KA, Leavens DA (2014) The importance of development for comparative primatology. Annu Rev Anthropol 43:183–200

    Article  Google Scholar 

  • Bard KA, Dunbar S, Maguire-Herring V, Veira Y, Hayes KG, McDonald K (2014) Gestures and social-emotional communicative development in chimpanzee infants. Am J Primatol 76:14–29

    Article  PubMed  Google Scholar 

  • Barrett L (2011) Beyond the brain: how body and environment shape animal and human minds. Princeton University Press, Princeton

    Google Scholar 

  • Bateson P (2017) Behaviour, development and evolution. Open Book Publishers, Cambridge

    Book  Google Scholar 

  • Boesch C (2007) What makes us human (Homo sapiens)? The challenge of cognitive cross-species comparison. J Comp Psychol 121:227–240

    Article  PubMed  Google Scholar 

  • Boesch C, Hohmann G, Marchant LF (2002) Behavioral diversity in chimpanzees and bonobos. Cambridge University Press, Cambridge

    Book  Google Scholar 

  • Bradbury JW, Vehrencamp SL (1998) Principles of animal communication. Sinauer, Sunderland

    Google Scholar 

  • Bretherton I, Bates E (1979) The emergence of intentional communication. New Dir Child Adolesc Dev 1979:81–100

    Article  Google Scholar 

  • Brumm H, Slabbekoorn H (2005) Acoustic communication in noise. Advances in the study of behavior, vol 35. Academic Press, London, pp 151–209. https://doi.org/10.1016/S0065-3454(05)35004-2

    Book  Google Scholar 

  • Byrne RW, Cartmill E, Genty E, Graham KE, Hobaiter C, Tanner J (2017) Great ape gestures: intentional communication with a rich set of innate signals. Anim Cogn 20:755–769

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  • Call J, Tomasello M (2007) The gestural communication of apes and monkeys. Lawrence Erlbaum Associates, New York

    Google Scholar 

  • Cartmill EA, Byrne RW (2007) Orangutans modify their gestural signaling according to their audience’s comprehension. Curr Biol 17:1345–1348

    Article  PubMed  CAS  Google Scholar 

  • Clay Z, Zuberbühler K (2011) Bonobos extract meaning from call sequences. PLoS One 6:e18786

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  • Cowley S (2011) Distributed language. John Benjamins, Amsterdam

    Book  Google Scholar 

  • Ekman P, Friesen W (1969) The repertoire of non-verbal behavior: categories, origins, usage and coding. Semiotica 1:49–98

    Article  Google Scholar 

  • Fröhlich M (2017a) Catarrhine communication. In: Vonk J, Shackelford TK (eds) Encyclopedia of animal cognition and behavior. Springer, Cham

    Google Scholar 

  • Fröhlich M (2017b) Taking turns across channels: conversation-analytic tools in animal communication. Neurosci Biobehav Rev 80:201–209

    Article  PubMed  Google Scholar 

  • Fröhlich M, Wittig RM, Pika S (2016) Should I stay or should I go? Initiation of joint travel in mother–infant dyads of two chimpanzee communities in the wild. Anim Cogn 19:483–500

    Article  PubMed  PubMed Central  Google Scholar 

  • Fröhlich M, Müller G, Zeiträg C, Wittig RM, Pika S (2017) Gestural development of chimpanzees in the wild: the impact of interactional experience. Anim Behav 134:271

    Article  Google Scholar 

  • Genty E, Clay Z, Hobaiter C, Zuberbühler K (2014) Multi-modal use of a socially directed call in Bonobos. PLoS One 9:e84738

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  • Ghazanfar AA (2013) Multisensory vocal communication in primates and the evolution of rhythmic speech. Behav Ecol Sociobiol 67:1441–1448

    Article  Google Scholar 

  • Ghazanfar AA, Rendall D (2008) Evolution of human vocal production vol 18, vol 11. Cell Press, Cambridge

    Google Scholar 

  • Gillespie-Lynch K, Greenfield PM, Lyn H, Savage-Rumbaugh S (2014) Gestural and symbolic development among apes and humans: support for a multimodal theory of language evolution. Front Psychol 5:1228

    Article  PubMed  PubMed Central  Google Scholar 

  • Goldin-Meadow S (1999) The role of gesture in communication and thinking. Trends Cogn Sci 3:419–429

    Article  PubMed  CAS  Google Scholar 

  • Goldin-Meadow S, Wagner SM (2005) How our hands help us learn. Trends Cogn Sci 9:234–241

    Article  PubMed  Google Scholar 

  • Gould SJ, Lewontin RC (1979) The spandrels of San Marco and the Panglossian paradigm: a critique of the adaptationist programme. Proc R Soc Lond B 205:581–598

    Article  PubMed  CAS  Google Scholar 

  • Graham JA, Argyle M (1975) A cross-cultural study of the communication of extra-verbal meaning by gestures. Int J Psychol 10:57–67

    Article  Google Scholar 

  • Graham KE, Furuichi T, Byrne RW (2016) The gestural repertoire of the wild bonobo (Pan paniscus): a mutually understood communication system. Anim Cogn 20:1–7

    Google Scholar 

  • Guilford T, Dawkins MS (1991) Receiver psychology and the evolution of animal signals. Anim Behav 42:1–14

    Article  Google Scholar 

  • Halfwerk W, Slabbekoorn H (2015) Pollution going multimodal: the complex impact of the human-altered sensory environment on animal perception and performance. Biol Lett 11:20141051

    Article  PubMed  PubMed Central  Google Scholar 

  • Halfwerk W, Page Rachel A, Taylor Ryan C, Wilson Preston S, Ryan Michael J (2014) Crossmodal comparisons of signal components allow for relative-distance assessment. Curr Biol 24:1751–1755

    Article  PubMed  CAS  Google Scholar 

  • Hauser MD, Teixidor P, Field L, Flaherty R (1993) Food-elicited calls in chimpanzees: effects of food quantity and divisibility? Anim Behav 45:817–819

    Article  Google Scholar 

  • Hebets EA, Papaj DR (2005) Complex signal function: developing a framework of testable hypotheses. Behav Ecol Sociobiol 57:197–214

    Article  Google Scholar 

  • Hebets EA, Vink CJ (2007) Experience leads to preference: experienced females prefer brush-legged males in a population of syntopic wolf spiders. Behav Ecol 18:1010–1020

    Article  Google Scholar 

  • Higham JP, Hebets EA (2013) An introduction to multimodal communication. Behav Ecol Sociobiol 67:1381–1388

    Article  Google Scholar 

  • Hobaiter C, Byrne RW (2011) Serial gesturing by wild chimpanzees: its nature and function for communication. Anim Cogn 14:827–838

    Article  PubMed  Google Scholar 

  • Hobaiter C, Byrne RW, Zuberbühler K (2017) Wild chimpanzees’ use of single and combined vocal and gestural signals. Behav Ecol Sociobiol 71:96

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  • Holle H, Gunter TC (2007) The role of iconic gestures in speech disambiguation: ERP evidence. J Cogn Neurosci 19:1175–1192

    Article  PubMed  Google Scholar 

  • Hopkins WD, Taglialatela JD, Leavens D (2007) Chimpanzees differentially produce novel vocalizations to capture the attention of a human. Anim Behav 73:281–286

    Article  PubMed  PubMed Central  Google Scholar 

  • Jawor JM, Gray N, Beall SM, Breitwisch R (2004) Multiple ornaments correlate with aspects of condition and behaviour in female northern cardinals, Cardinalis cardinalis. Anim Behav 67:875–882

    Article  Google Scholar 

  • Johnstone RA (1996) Multiple displays in animal communication: ‘backup signals’ and ‘multiple messages’. Philos Trans R Soc Lond B Biol Sci 351:329–338

    Article  Google Scholar 

  • Katsu N, Yamada K, Nakamichi M (2017) Influence of social interactions with nonmother females on the development of call usage in Japanese macaques. Anim Behav 123:267–276

    Article  Google Scholar 

  • Kelly SD, Barr DJ, Church R, Lynch K (1999) Offering a hand to pragmatic understanding: the role of speech and gesture in comprehension and memory. J Mem Lang 40:577–592

    Article  Google Scholar 

  • Kendon A (2004) Gesture: visible action as utterance. Cambridge University Press, Cambridge

    Book  Google Scholar 

  • King BJ, Shanker SG (2003) How can we know the dancer from the dance: the dynamic nature of African great ape social communication. Anthropol Theory 3:5–23

    Article  Google Scholar 

  • Laporte MNC, Zuberbühler K (2011) The development of a greeting signal in wild chimpanzees. Dev Sci 14:1220–1234

    Article  PubMed  Google Scholar 

  • Leavens DA, Bard KA (2011) Environmental influences on joint attention in great apes: implications for human cognition. J Cogn Educ Psychol 10:9–31

    Article  Google Scholar 

  • Leavens DA, Hopkins WD (2005) Multimodal concomitants of manual gesture by chimpanzees: influence of food size and distance. Gesture 5:73–88

    Article  Google Scholar 

  • Leavens DA, Russell JL, Hopkins WD (2005) Intentionality as measured in the persistence and elaboration of communication by chimpanzees (Pan troglodytes). Child Dev 76:291–306

    Article  PubMed  PubMed Central  Google Scholar 

  • Leavens DA, Russell J, Hopkins W (2010) Multimodal communication by captive chimpanzees (Pan troglodytes). Anim Cogn 13:33–40

    Article  PubMed  Google Scholar 

  • Levinson SC (2016) Turn-taking in human communication–origins and implications for language processing. Trends Cogn Sci 20:6–14

    Article  PubMed  Google Scholar 

  • Levinson SC, Holler J (2014) The origin of human multi-modal communication. Philos Trans R Soc Lond B Biol Sci 369:20130302

    Article  PubMed  PubMed Central  Google Scholar 

  • Liebal K, Pika S, Tomasello M (2006) Gestural communication of orangutans (Pongo pygmaeus). Gesture 6:1–38

    Article  Google Scholar 

  • Liebal K, Waller BM, Burrows AM, Slocombe KE (2014) Primate communication: a multimodal approach. Cambridge University Press, Cambridge

    Google Scholar 

  • Macaluso E, Frith CD, Driver J (2000) Modulation of human visual cortex by crossmodal spatial attention. Science 289:1206–1208

    Article  PubMed  CAS  Google Scholar 

  • Marler P (1961) The logical analysis of animal communication. J Theor Biol 1:295–317

    Article  PubMed  CAS  Google Scholar 

  • Massaro DW (1998) Perceiving talking faces: from speech perception to a behavioral principle. MIT Press, Cambridge

    Google Scholar 

  • McGurk H, MacDonald J (1976) Hearing lips and seeing voices. Nature 264:746–748

    Article  PubMed  CAS  Google Scholar 

  • McNeill D (1992) Hand and mind: what gestures reveal about thought. University of Chicago Press, Chicago

    Google Scholar 

  • Micheletta J, Engelhardt A, Matthews LEE, Agil M, Waller BM (2013) Multicomponent and multimodal lipsmacking in crested macaques (Macaca nigra). Am J Primatol 75:763–773

    Article  PubMed  Google Scholar 

  • Michelsen A, Andersen BB, Storm J, Kirchner WH, Lindauer M (1992) How honeybees perceive communication dances, studied by means of a mechanical model. Behav Ecol Sociobiol 30:143–150

    Article  Google Scholar 

  • Møller A, Pomiankowski A (1993) Why have birds got multiple sexual ornaments? Behav Ecol Sociobiol 32:167–176

    Google Scholar 

  • Munoz NE, Blumstein DT (2012) Multisensory perception in uncertain environments. Behav Ecol 23:457–462

    Article  Google Scholar 

  • Partan SR (2002) Single and multichannel signal composition: facial expressions and vocalizations of rhesus macaques (Macaca mulatta). Behaviour 139:993–1027

    Article  Google Scholar 

  • Partan SR (2013) Ten unanswered questions in multimodal communication. Behav Ecol Sociobiol 67:1523–1539

    Article  PubMed  PubMed Central  Google Scholar 

  • Partan SR (2016) Multimodal shifts in noise: switching channels to communicate through rapid environmental change. Anim Behav 124:325

    Article  Google Scholar 

  • Partan SR, Marler P (1999) Communication goes multimodal. Science 283:1272–1273

    Article  PubMed  CAS  Google Scholar 

  • Partan SR, Marler P (2005) Issues in the classification of multimodal communication signals. Am Nat 166:231–245

    Article  PubMed  Google Scholar 

  • Pika S, Fröhlich M (2018) Gestural acquisition in great apes: the social negotiation hypothesis. Anim Cogn 1:15

    Google Scholar 

  • Pika S, Liebal K, Tomasello M (2003) Gestural communication in young gorillas (Gorilla gorilla): gestural repertoire, learning, and use. Am J Primatol 60:95–111

    Article  PubMed  Google Scholar 

  • Pika S, Liebal K, Tomasello M (2005) Gestural communication in subadult bonobos (Pan paniscus): repertoire and use. Am J Primatol 65:39–61

    Article  PubMed  Google Scholar 

  • Pollick AS, de Waal FBM (2007) Ape gestures and language evolution. Proc Natl Acad Sci 104:8184–8189

    Article  PubMed  CAS  Google Scholar 

  • Rowe C (1999a) One signal or two? Science 284:741 h

    Article  Google Scholar 

  • Rowe C (1999b) Receiver psychology and the evolution of multicomponent signals. Anim Behav 58:921–931

    Article  PubMed  CAS  Google Scholar 

  • Ruxton G, Schaefer H (2011) Resolving current disagreements and ambiguities in the terminology of animal communication. J Evol Biol 24:2574–2585

    Article  PubMed  CAS  Google Scholar 

  • Rybak F, Sureau G, Aubin T (2002) Functional coupling of acoustic and chemical signals in the courtship behaviour of the male Drosophila melanogaster. Proc R Soc Lond B Biol Sci 269:695–701

    Article  CAS  Google Scholar 

  • Scott-Phillips TC, Dickins TE, West SA (2011) Evolutionary theory and the ultimate–proximate distinction in the human behavioral sciences. Perspect Psychol Sci 6:38–47

    Article  PubMed  Google Scholar 

  • Shanker SG, King BJ (2002) The emergence of a new paradigm in ape language research: beyond interactionism. Behav Brain Sci 25:646–651

    Google Scholar 

  • Slocombe KE, Waller BM, Liebal K (2011) The language void: the need for multimodality in primate communication research. Anim Behav 81:919–924

    Article  Google Scholar 

  • Smith WJ (1965) Message, meaning, and context in ethology. Am Nat 99:405–409

    Article  Google Scholar 

  • Snowdon CT, Hausberger M (eds) (1997) Social influences on vocal development. Cambridge University Press, Cambridge

    Google Scholar 

  • Soto-Faraco S, Calabresi M, Navarra J, Werker J, Lewkowicz DJ (2012) The development of audiovisual speech perception. Multisensory Development. Oxford University Press, Oxford, pp 207–228

    Google Scholar 

  • Spence C, Nicholls ME, Gillespie N, Driver J (1998) Cross-modal links in exogenous covert spatial orienting between touch, audition, and vision. Percept Psychophys 60:544–557

    Article  PubMed  CAS  Google Scholar 

  • Sumby WH, Pollack I (1954) Visual contribution to speech intelligibility in noise. J Acoust Soc Am 26:212–215

    Article  Google Scholar 

  • Taborsky B, Oliveira RF (2012) Social competence: an evolutionary approach. Trends Ecol Evol 27:679–688

    Article  PubMed  Google Scholar 

  • Taglialatela JP, Russell JL, Schaeffer JA, Hopkins WD (2011) Chimpanzee vocal signaling points to a multimodal origin of human language. PLoS One 6:e18852

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  • Taglialatela JP et al (2015) Multimodal communication in chimpanzees. Am J Primatol 77:1143–1148

    Article  PubMed  PubMed Central  Google Scholar 

  • Tinbergen N (1963) On aims and methods of ethology. Ethology 20:410–433

    Google Scholar 

  • Tomasello M (2008) Origins of human communication, vol 2008. MIT Press, Cambridge

    Google Scholar 

  • Tomasello M, Call J, Nagell K, Olguin R, Carpenter M (1994) The learning and use of gestural signals by young chimpanzees: a trans-generational study. Primates 35:137–154

    Article  Google Scholar 

  • Tomasello M, Call J, Warren J, Frost T, Carpenter M, Nagell K (1997) The ontogeny of chimpanzee gestural signals: a comparison across groups and generations. Evol Commun 1:224–259

    Google Scholar 

  • Townsend SW et al (2016) Exorcising Grice’s ghost: an empirical approach to studying intentional communication in animals. Biol Rev 92:1427–1433

    Article  PubMed  Google Scholar 

  • van Schaik CP (2004) Among orangutans: red apes and the rise of human culture. Belknap, Cambridge

    Google Scholar 

  • Van Schaik CP (2016) The primate origins of human nature. Wiley, Oxford

    Google Scholar 

  • Vigliocco G, Perniss P, Vinson D (2014) Language as a multimodal phenomenon: implications for language learning, processing and evolution. The Royal Society, London

    Google Scholar 

  • Waller BM, Liebal K, Burrows AM, Slocombe KE (2013) How can a multimodal approach to primate communication help us understand the evolution of communication? Evol Psychol 11:147470491301100305

    Article  Google Scholar 

  • Wedekind C, Meyer P, Frischknecht M, Niggli UA, Pfander H (1998) Different carotenoids and potential information content of red coloration of male three-spined stickleback. J Chem Ecol 24:787–801

    Article  CAS  Google Scholar 

  • Wheeler BC, Fischer J (2012) Functionally referential signals: a promising paradigm whose time has passed. Evol Anthropol Issues News Rev 21:195–205

    Article  Google Scholar 

  • Wilke C, Kavanagh E, Donnellan E, Waller BM, Machanda ZP, Slocombe KE (2017) Production of and responses to unimodal and multimodal signals in wild chimpanzees, Pan troglodytes schweinfurthii. Anim Behav 123:305–316

    Article  Google Scholar 

  • Wilson EO (1976) Sociobiology. Belknap, Cambridge

    Google Scholar 

  • Zuk M, Ligon JD, Thornhill R (1992) Effects of experimental manipulation of male secondary sex characters on female mate preference in red jungle fowl. Anim Behav 44:999–1006

    Article  Google Scholar 

Download references

Acknowledgements

We thank Wouter Halfwerk, Maria van Noordwijk, David Leavens and Thibaud Gruber for fruitful discussions, and Kim Bard and two anonymous reviewers for useful comments on the manuscript.

Funding

M.F. was funded by the Sponsorship Society of the German Primate Centre, the A.H. Schultz Foundation and the Forschungskredit of the University of Zurich, Grant no. FK-17-106.

Author information

Authors and Affiliations

  1. Department of Anthropology, University of Zurich, Winterthurerstrasse 190, 8057, Zurich, Switzerland

    Marlen Fröhlich & Carel P. van Schaik

Authors
  1. Marlen Fröhlich
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carel P. van Schaik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marlen Fröhlich.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fröhlich, M., van Schaik, C.P. The function of primate multimodal communication. Anim Cogn 21, 619–629 (2018). https://doi.org/10.1007/s10071-018-1197-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10071-018-1197-8

Keywords

Search

Navigation