Despite rich behavioral evidence, it is unclear how the brain expands its behavior repertoire. By building theoretical models with a deep reinforcement learning algorithm, I show that the brain composes a behavior to solve a novel task by combining previously acquired skills and augmenting their variability.
This is a preview of subscription content, log in via an institution to check access.
References
Kohler, W. The Mentality of Apes (trans Winter, E.) (Liveright, 1976). This book presents the intelligent behavior repertoires of chimpanzees.
Epstein, R., Kirshnit, C. E., Lanza, R. P. & Rubin, L. C. ‘Insight’ in the pigeon: antecedents and determinants of an intelligent performance. Nature 308, 61–62 (1984). This paper reports the sudden acquisition of a problem solution in pigeons by combining pre-learned motor skills.
Arthur, W. B. The Nature of Technology: What It is and How It Evolves (Free Press, 2009). This book presents the concept of combinatorial evolution of technology.
Botvinick, M., Wang, J. X., Dabney, W., Miller, K. J. & Kurth-Nelson, Z. Deep reinforcement learning and its neuroscientific implications. Neuron 107, 603–616 (2020). This paper emphasizes the benefits of collaborative efforts between deep reinforcement learning and neuroscience.
Haarnoja, T. et al. Composable deep reinforcement learning for robotic manipulation. 2018 IEEE International Conf. on Robotics and Automation (ICRA) 6244–6251 (2018). This paper reports a deep reinforcement learning-based theoretical framework for combinatorial policy composition in machines.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This is a summary of: Makino, H. Arithmetic value representation for hierarchical behavior composition. Nat. Neurosci. https://doi.org/10.1038/s41593-022-01211-5 (2022).
Rights and permissions
About this article
Cite this article
Deep reinforcement learning to study combinatorial expansion of a behavior repertoire. Nat Neurosci 26, 2–3 (2023). https://doi.org/10.1038/s41593-022-01235-x
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41593-022-01235-x
- Springer Nature America, Inc.