All of us have marveled at the remarkable diversity of animal behaviors in nature.
None of us has much idea of how these have evolved.
Like other inherited phenotypes, many behavioral traits of animals—predatory instincts, courtship rituals, and shelter building, to name but a few—have a genetic basis (Fig. 1). Genes, of course, don’t control behavior directly, but encode the vast array of molecules that establish the connectivity and physiology of the nervous system (to make no mention of those that form the tissues and organs in which neural circuits are embedded). What is the genetic basis by which seemingly complex behaviors have evolved?
This grand question covers a multitude of issues. Is there a subset of genes that have “special” roles in dictating behavioral evolution, or does behavior evolve along many different trajectories? What is the relative contribution of structural and regulatory genetic changes (influencing protein function and gene expression, respectively) to this process? What part(s) of the nervous system do such genetic changes impact: sensory input channels, central processing circuit and/or in locomotor pathways that directly control actions? Do answers to these questions vary among taxa?
Why are these questions timely?
Technical and intellectual reasons. We’re getting very good at observing and quantifying behavior. Cameras and computers do much of the job, revealing behaviors that previously escaped detection by the human eye or which would be too laborious to measure manually . With large-scale population surveys and comparative genomics, we’re also more attuned to the intra- and interspecific behavioral  and genetic variation  that exists around the world. We also simply now know a lot about the genetic and neural basis of behavior of well-established genetic model systems, such as the mouse Mus musculus, the fly Drosophila melanogaster and the worm Caenorhabditis elegans. These models provide valuable points of comparison with related species that display obvious (and presumably evolutionarily significant) variants in their behaviors. Importantly, with new genome editing methods (such as CRISPR/Cas9 ) and optogenetic and thermogenetic tools for turning on and off neurons at will , we’re potentially able to manipulate molecules and circuits with precision in such non-traditional model species. This ability will allow us to determine causal relationships—and not simply correlations—between genetic and behavioral variation.
Why should neurobiologists care about these questions?
Despite impressive advances, we’re still a long way from understanding the genetic and neural basis of even simple behaviors. Seymour Benzer’s seminal work in the 1960s with D. melanogaster showed how forward genetics can identify key components of specific behaviors, such as courtship or circadian activity . Despite the doors this work opened, the opposing view of Benzer’s contemporary, Jerry Hirsch, that behaviors are too complex to be reducible to the action of single (or a few) genes, is of course largely true. In flies and worms, large-scale genetic screens for loci underlying, for example, embryonic segmentation or axon guidance have been extraordinarily fruitful . By contrast, behavioral screens are much harder to perform and, because there are many uninteresting ways in which a particular behavior can be disrupted, truly informative mutations are likely to be rare. Indeed, most characterized examples affect genes that have non-pleiotropic functions in peripheral sensory systems, such as specific olfactory receptors [2, 8]. Thus, a comparative, evolutionary approach that takes advantage of naturally occurring, phenotypically consequent genetic variants can offer a complementary way to identify molecular determinants of behavior. These may open new doors into explorations of the underlying neural circuits.
Why should evolutionary biologists care about these questions?
Although behavioral traits are critical to animals’ survival and reproduction, the challenge of high-throughput and robust quantification of these often-complex phenotypes has hampered exploration of their evolution, especially by comparison with studies of, for example, morphological characteristics . This is changing: the current intersection of phenotyping and genomic technologies is rapidly increasing our ability to link behavioral variation with specific regions of the genome, through quantitative trait locus and association mapping . Importantly, these advances not only allow determination of the genetic architecture of behavioral variations (as in burrow building by Peromyscus mice; Fig. 1) ), but can also allow us to nail the causal gene(s) and genetic variant(s), as exemplified by studies of drosophilid courtship song  and C. elegans’ sensitivity to environmental gases [12, 13].
These tools can also enrich our ability to understand the evolutionary processes that govern behavioral divergence. Reverse genetic approaches, in which regions of the genome are identified because they carry signatures of selection, are often a starting point for this endeavor . While the behavioral impact of such candidate regions has rarely been characterized experimentally, deepened knowledge of nervous systems and technical advances now make hypothesis generation and testing much more easily achieved. The increased ability to integrate evolutionary processes (e.g., mutation, selection, genetic drift) with neurobiological mechanisms will translate into a richer understanding of behavioral divergence. This, in turn, may help us understand to what extent selective versus non-selective forces are responsible for the evolution of behavior.
What questions (and challenges) are there in the long-term?
Most initial traction on the problem of behavioral evolution is likely to come from studies of simple innate actions that can be reproduced in the laboratory. However, once efficient assays are established, we can begin to tackle behaviors that are shaped by internal state and experience, in essence trying to understand the evolutionary plasticity of neural plasticity. What remains a looming challenge is to place our favorite behavioral differences within an ecological context and understand the impact of these differences on fitness. This will be difficult for many reasons: fitness effects may be extremely small (though still impactful on an evolutionary scale) and many of our current model species’ behaviors in nature are poorly appreciated. Insights may thus only come from large-scale, field-based studies. Serious consideration should also be given to concerted efforts to establish new model species  which retain a close relationship with current model systems but have better-understood ecologies.
In his landmark book , Darwin opened his chapter on “Instinct” with a characteristically cautious statement: “I have nothing to do with the origin of the primary mental powers, any more than I have with that of life itself”. Over 150 years later, we might now be cautiously optimistic that an understanding of the evolution of at least some “mental powers” is within our reach.
Robie AA, Seagraves KM, Egnor SE, Branson K. Machine vision methods for analyzing social interactions. J Exp Biol. 2017;220(Pt 1):25–34.
Bendesky A, Bargmann CI. Genetic contributions to behavioural diversity at the gene-environment interface. Nat Rev Genet. 2011;12(12):809–20.
Savolainen O, Lascoux M, Merila J. Ecological genomics of local adaptation. Nat Rev Genet. 2013;14(11):807–20.
Hsu PD, Lander ES, Zhang F. Development and applications of CRISPR-Cas9 for genome engineering. Cell. 2014;157(6):1262–78.
Bernstein JG, Garrity PA, Boyden ES. Optogenetics and thermogenetics: technologies for controlling the activity of targeted cells within intact neural circuits. Curr Opin Neurobiol. 2012;22(1):61–71.
Weiner J. Time, love, memory: a great biologist and his quest for the origins of behavior. New York. London: Knopf; 1999.
St Johnston D. The art and design of genetic screens: Drosophila melanogaster. Nat Rev Genet. 2002;3(3):176–88.
Cande J, Prud’homme B, Gompel N. Smells like evolution: the role of chemoreceptor evolution in behavioral change. Curr Opin Neurobiol. 2012;23(1):152–8.
Stern DL. Evolution, development and the predictable genome. Greenwood Village (Colorado): Roberts and Company Publishers; 2011.
Hu CK, Hoekstra HE. Peromyscus burrowing: a model system for behavioral evolution. Semin Cell Dev Biol. 2017;61:107–14.
Ding Y, Berrocal A, Morita T, Longden KD, Stern DL. Natural courtship song variation caused by an intronic retroelement in an ion channel gene. Nature. 2016;536(7616):329–32.
Persson A, Gross E, Laurent P, Busch KE, Bretes H, de Bono M. Natural variation in a neural globin tunes oxygen sensing in wild Caenorhabditis elegans. Nature. 2009;458(7241):1030–3.
McGrath PT, Rockman MV, Zimmer M, Jang H, Macosko EZ, Kruglyak L, Bargmann CI. Quantitative mapping of a digenic behavioral trait implicates globin variation in C. elegans sensory behaviors. Neuron. 2009;61(5):692–9.
Vitti JJ, Grossman SR, Sabeti PC. Detecting natural selection in genomic data. Annu Rev Genet. 2013;47:97–120.
Sommer RJ. The future of evo-devo: model systems and evolutionary theory. Nat Rev Genet. 2009;10(6):416–22.
Darwin C. On the origin of species by means of natural selection, or the preservation of favoured races in the struggle for life. London: John Murray; 1859.
Cande J, Andolfatto P, Prud’homme B, Stern DL, Gompel N. Evolution of multiple additive loci caused divergence between Drosophila yakuba and D. santomea in wing rowing during male courtship. PLoS One. 2012;7(8):e43888.
We are grateful to Nicolas Gompel, Benjamin Prud’homme, Brant Peterson, and Hopi Hoekstra for providing images. We thank Thomas Auer, Margarida Cardoso-Moreira, Sophie Martin, and Lucia Prieto-Godino for comments on the manuscript. JRA was supported by a post-doctoral fellowship from Novartis Foundation for medical-biological Research (12A14). Research in RB’s laboratory is supported by the University of Lausanne, an ERC Consolidator Grant (615094), and the Swiss National Science Foundation.
JRA and RB wrote the article. Both authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Arguello, J.R., Benton, R. Open questions: Tackling Darwin’s “instincts”: the genetic basis of behavioral evolution. BMC Biol 15, 26 (2017). https://doi.org/10.1186/s12915-017-0369-3