Behavioral Ecology and Sociobiology

, Volume 67, Issue 9, pp 1381–1388 | Cite as

An introduction to multimodal communication

  • James P. HighamEmail author
  • Eileen A. Hebets
Original Paper


Though it has long been known that animal communication is complex, recent years have seen growing interest in understanding the extent to which animals give multicomponent signals in multiple modalities, and how the different types of information extracted by receivers are interpreted and integrated in animal decision-making. This interest has culminated in the production of the present special issue on multimodal communication, which features both theoretical and empirical studies from leading researchers in the field. Reviews, comparative analyses, and species-specific empirical studies include manuscripts on taxa as diverse as spiders, primates, birds, lizards, frogs, and humans. The present manuscript serves as both an introduction to this special issue, as well as an introduction to multimodal communication more generally. We discuss the history of the study of complexity in animal communication, issues relating to defining and classifying multimodal signals, and particular issues to consider with multimodal (as opposed to multicomponent unimodal) communication. We go on to discuss the current state of the field, and outline the contributions contained within the issue. We finish by discussing future avenues for research, in particular emphasizing that ‘multimodal’ is more than just ‘bimodal’, and that more integrative frameworks are needed that incorporate more elements of efficacy, such as receiver sensory ecology and the environment.


Multimodal communication Animal signals Receiver sensory perception 



We are very grateful to Esteban Fernandez-Juricic and two anonymous reviewers for comments on a previous draft of this manuscript. We extend our special thanks to Theo Bakker, James Traniello, and Saskia Hesse for their considerable help with the production of this special issue.


  1. Akre KL, Ryan MJ (2010) Complexity increases working memory for mating signals. Curr Biol 20:502–505PubMedCrossRefGoogle Scholar
  2. Andersson M (1994) Sexual selection. Princeton University Press, PrincetonGoogle Scholar
  3. Boughman JW (2001) Divergent sexual selection enhances reproductive isolation in sticklebacks. Nature 411:944–948PubMedCrossRefGoogle Scholar
  4. Bro-Jorgensen J (2010) Dynamics of multiple signalling systems: animal communication in a world in flux. Trends Ecol Evol 25:292–300PubMedCrossRefGoogle Scholar
  5. Bushmann PJ (1999) Concurrent signals and behavioral plasticity in blue crab (Callinectes sapiduss Rathburn) courtship. Biol Bull 197:63–71CrossRefGoogle Scholar
  6. Candolin U (2003) The use of multiple cues in mate choice. Biol Rev 78:575–595PubMedCrossRefGoogle Scholar
  7. Dambly-Chaudière C, Sapède D, Soubrian F, Decorde K, Gompel N, Ghysen A (2003) The lateral line of zebrafish, a model system for the analysis of morphogenesis and neural development in vertebrates. Biol Cell 95:579–587PubMedCrossRefGoogle Scholar
  8. Darwin C (1872) The expression of the emotions in man and animals. John Murray, LondonCrossRefGoogle Scholar
  9. Elias DO, Mason AC, Maddison WP, Hoy RR (2003) Seismic signals in a courting male jumping spider (Araneae: Salticidae). J Exp Biol 206:4029–4039PubMedCrossRefGoogle Scholar
  10. Elias DO, Lee N, Hebets EA, Mason AC (2006) Seismic signal production in a wolf spider: parallel versus serial multicomponent signals. J Exp Biol 209:1074–1084PubMedCrossRefGoogle Scholar
  11. Elias DO, Maddison WP, Peckmezian C, Girard MB, Mason AC (2012) Orchestrating the score: complex multimodal courtship in the Coecatus-group of Habronattus (Araneae: Salticidae). Biol J Linn Soc 105:522–547CrossRefGoogle Scholar
  12. Ghazanfar AA (2013) Multisensory vocal communication in primates and the evolution of rhythmic speech. Behav Ecol Sociobiol. doi: 10.1007/s00265-013-1491-z Google Scholar
  13. Ghazanfar AA, Logothetis NK (2003) Facial expressions linked to monkey calls. Nature 423:937–938PubMedCrossRefGoogle Scholar
  14. Ghazanfar AA, Schroeder CE (2006) Is the neocortex essentially multisensory? Trends Cogn Sci 10:278–285PubMedCrossRefGoogle Scholar
  15. Gleeson RA (1991) Intrinsic factors mediating pheromone communication in the blue crab, Callinectes sapidus. In: Martin JW, Bauer RT (eds) Crustacean sexual biology. Columbia University Press, New York, pp 17–32Google Scholar
  16. Grafe TU, Preininger D, Sztatecsny M, Kasah R, Dehling JM, Proksch S, Hödl W (2012) Multimodal communication in a noisy environment: a case study of the Bornean rock frog Staurois parvus. PLoS One 7:e37965PubMedCrossRefGoogle Scholar
  17. Hebets EA (2005) Attention-altering signal interactions in the multimodal courtship display of the wolf spider Schizocosa uetzi. Behav Ecol 16:75–82CrossRefGoogle Scholar
  18. Hebets EA (2011) Current status and future directions of research in complex signaling. Curr Zool 57:i–vGoogle Scholar
  19. Hebets EA, Papaj DR (2005) Complex signal function: developing a testable framework for testable hypotheses. Behav Ecol Sociobiol 57:197–214CrossRefGoogle Scholar
  20. Hebets EA, Rundus AS (2011) Chemical communication in a multimodal context. In: Breithaupt T, Thiel M (eds) Chemical communication in crustaceans. Springer, New York, pp 335–354Google Scholar
  21. Hebets EA, Vink CJ (2007) Experience leads to preference: experienced females prefer brush-legged males in a population of syntopic wolf spiders. Behav Ecol 18:1010–1020CrossRefGoogle Scholar
  22. Hebets EA, Vink CJ, Sullivan-Beckers L, Rosenthal MF (2013) The dominance of seismic signaling and selection for signal complexity in Schizocosa multimodal courtship displays. Behav Ecol and Sociobiol.
  23. Heymann EW (2006) The neglected sense—olfaction in primate behavior, ecology, and evolution. Am J Primatol 68:519–524PubMedCrossRefGoogle Scholar
  24. Higham JP, Hughes KD, Brent LJN, Dubuc C, Engelhardt A, Heistermann M, Maestripieri D, Santos LR, Stevens M (2011) Familiarity affects assessment of facial signals of female fertility by free-ranging male rhesus macaques. Proc R Soc Lond B 278:3452–3458CrossRefGoogle Scholar
  25. Higham JP, Pfefferle D, Heistermann M, Maestripieri D, Stevens M (2013) Signaling in multiple modalities in male rhesus macaques: sex skin coloration and barks in relation to androgen levels, social status, and mating behavior. Behav Ecol Sociobiol. doi: 10.1007/s00265-013-1521-x Google Scholar
  26. Johnstone RA (1996) Multiple displays in animal communication: 'backup signals' and 'multiple messages'. Philos T Roy Soc B 351:329–338CrossRefGoogle Scholar
  27. Kamio M, Reidenbach MA, Derby CD (2008) To paddle or not: context dependent courtship display by male blue crabs, Callinectes sapidus. J Exp Biol 211:1243–1248PubMedCrossRefGoogle Scholar
  28. Kohl JV, Atzmueller M, Fink B, Grammer K (2001) Human pheromones: integrating neuroendocrinology and ethology. Neuro Endocrinol Lett 22:309–321PubMedGoogle Scholar
  29. Lovelace CT, Stein BE, Wallace MT (2003) An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection. Cogn Brain Res 17:447–453CrossRefGoogle Scholar
  30. Maan ME, Seehausen O (2010) Mechanisms of species divergence through visual adaptation and sexual selection – perspectives from a cichlid model system. Curr Zool 56:285–299Google Scholar
  31. Marler P (1961) The logical analysis of animal communication. J Theor Biol 1:295–317PubMedCrossRefGoogle Scholar
  32. Marler P (1965) Communication in monkeys and apes. In: DeVore I (ed) Primate behavior: field studies of monkeys and apes. Holt, Rinehart and Winston, New York, pp 544–584Google Scholar
  33. Maynard Smith J, Harper DGC (2003) Animal signals. Oxford University Press, OxfordGoogle Scholar
  34. McGurk H, MacDonald J (1976) Hearing lips and seeing voices. Nature 264:746–748PubMedCrossRefGoogle Scholar
  35. Munoz NE, Blumstein DT (2012) Multisensory perception in uncertain environments. Behav Ecol 23:457–462CrossRefGoogle Scholar
  36. Ossip-Klein AG, Fuentes JA, Hews DK, Martins EP (2013) Information content is more important than sensory system or physical distance in guiding the long-term evolutionary relationships between signaling modalities in Sceloporus lizards. Behav Ecol and Sociobiol.
  37. Partan SR, Marler P (1999a) Communication goes multimodal. Science 283:1272–1273PubMedCrossRefGoogle Scholar
  38. Partan SR, Marler P (1999b) Response to Rowe. Science 284:744Google Scholar
  39. Partan SR, Marler P (2005) Issues in the classification of multisensory communication signals. Am Nat 166:231–245PubMedCrossRefGoogle Scholar
  40. Partan SR (2013) Ten unanswered questions in multimodal communication. Behav Ecol Sociobiol. doi: 10.1007/s00265-013-1565-y
  41. Preininger D, Boeckle M, Freudmann A, Starnberger I, Sztatecsny M, Hödl W (2013) Multimodal signaling in the Small Torrent Frog (Micrixalus saxicola) in a complex acoustic environment. Behav Ecol Sociobiol. doi: 10.1007/s00265-013-1489-6 Google Scholar
  42. Price TD (2006) Phenotypic plasticity, sexual selection and the evolution of colour patterns. J Exp Biol 209:2368–2376PubMedCrossRefGoogle Scholar
  43. Renoult JP, Schaefer HM, Sallé B, Charpantier MJE (2011) The evolution of the multicoloured face of mandrills: insights from the perceptual space of colour vision. PLoS One 6:e29117PubMedCrossRefGoogle Scholar
  44. Rodríguez RL, Cocroft RB (2006) Divergence in female duetting signals in the Enchenopa binotata species complex of treehoppers (Hemiptera: Membracidae). Ethology 112:1231–1238CrossRefGoogle Scholar
  45. Rodríguez RL, Ramaswamy K, Cocroft RB (2006) Evidence that female preferences have shaped male signal evolution in a clade of specialized plant-feeding insects. Proc R Soc Lond B 273:2585–2593CrossRefGoogle Scholar
  46. Ronald KL, Fernández-Juricic E, Lucas JR (2012) Taking the sensory approach: how individual differences in sensory perception can influence mate choice. Anim Behav 84:1283–1294CrossRefGoogle Scholar
  47. Rowe C (1999) One signal or two? Science 284:743–744PubMedCrossRefGoogle Scholar
  48. Rowe C, Guildford T (1996) Hidden colour aversions in domestic chicks triggered by pyrazine odours of insect warning displays. Nature 383:520–522CrossRefGoogle Scholar
  49. Rowe C, Halpin C (2013) Why are warning displays multimodal? Behav Ecol Sociobiol. doi: 10.1007/s00265-013-1515-8 Google Scholar
  50. Ruxton GD, Schaefer HM (2011) Resolving current disagreements and ambiguities in the terminology of animal communication. J Evol Biol 24:2574–2585PubMedCrossRefGoogle Scholar
  51. Semple S, Gerald M, Suggs D (2009) Bystanders affect the outcome of mother-infant interactions in rhesus macaques. Proc R Soc Lond B 276:2257–2262CrossRefGoogle Scholar
  52. Slocombe KE, Zuberbühler K (2007) Chimpanzees modify recruitment screams as a function of audience composition. Proc Natl Acad Sci U S A 104:17228–17233PubMedCrossRefGoogle Scholar
  53. Slocombe KE, Kaller T, Turman L, Townsend SW, Papworth S, Zuberbühler K (2010) Production of food-associated calls in wild male chimpanzees is dependent on the composition of the audience. Behav Ecol Sociobiol 64:1959–1966CrossRefGoogle Scholar
  54. Smith WJ (1977) The behavior of communicating: an ethological approach. Harvard University Press, CambridgeGoogle Scholar
  55. Smith CL, Evans CS (2013) A new heuristic for capturing the complexity of multimodal signals. Behav Ecol Sociobiol. doi: 10.1007/s00265-013-1490-0 Google Scholar
  56. Sorenson MD, Sefc KM, Payne RB (2003) Speciation by host switch in brood parasitic indigobirds. Nature 424:928–931PubMedCrossRefGoogle Scholar
  57. Stafstrom JA, Hebets EA (2013) Female mate choice for multimodal courtship and the importance of the signaling background for selection on male ornamentation. Curr Zool 59:200–209Google Scholar
  58. Stein BE (ed) (2012) The new handbook of multisensory integration. Massachusetts Institute of Technology Press, CambridgeGoogle Scholar
  59. Stevens M (2013) Sensory ecology, behavior and evolution. Cambridge University Press, CambridgeGoogle Scholar
  60. Taylor RC, Klein BA, Stein J, Ryan MJ (2008) Faux frogs: multi-modal signalling and the value of robotics in animal behavior. Anim Behav 76:1089–1097CrossRefGoogle Scholar
  61. Taylor RC, Klein BA, Stein J, Ryan MJ (2011) Multimodal signal variation in space and time: how important is matching a signal with its signaler? J Exp Biol 214:815–820PubMedCrossRefGoogle Scholar
  62. Uetz GW, Stratton GE (1983) Communication in spiders. Endeavour 7:13–18CrossRefGoogle Scholar
  63. Uetz GW, Roberts JA, Clark DL, Gibson JS, Gordon SD (2013) Multimodal signals increase active space of communication by wolf spiders in a complex litter environment. Behav Ecol Sociobiol. doi: 10.1007/s00265-013-1557-y Google Scholar
  64. Uy JAC, Safran RJ (2013) Variation in the temporal and spatial use of signals and its implications for multimodal communication. Behav Ecol Sociobiol. doi: 10.1007/s00265-013-1492-y Google Scholar
  65. VanderSal ND, Hebets EA (2007) Cross-modal effects on learning: a seismic stimulus improves color discrimination learning in a jumping spider. J Exp Biol 210:3689–3695PubMedCrossRefGoogle Scholar
  66. Wilgers D, Hebets EA (2011) Complex courtship displays facilitate male reproductive success and plasticity in signaling across variable environments. Curr Zool 57:175–186Google Scholar
  67. Wilson AJ, Dean M, Higham JP (2013) A game theoretic approach to multimodal communication. Behav Ecol Sociobiol. doi: 10.1007/s00265-013-1589-3 Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.Department of AnthropologyNew York UniversityNew YorkUSA
  2. 2.School of Biological SciencesUniversity of NebraskaKearneyUSA

Personalised recommendations