Abstract
Associative learning and memory, i.e., learning and remembering the associations between environmental stimuli, self-generated actions, and outcomes such as rewards or punishments, are critical for the well-being of animals. Hence, the neural mechanisms underlying these processes are extensively studied using behavioral tasks in laboratory animals. Traditionally, these tasks have been controlled using commercial hardware and software, which limits scalability and accessibility due to their cost. More recently, due to the revolution in microcontrollers or microcomputers, several general-purpose and open-source solutions have been advanced for controlling neuroscientific behavioral tasks. While these solutions have great strength due to their flexibility and general-purpose nature, for the same reasons, they suffer from some disadvantages including the need for considerable programming expertise, limited online visualization, or slower than optimal response latencies for any specific task. Here, to mitigate these concerns, we present an open-source behavior controller for associative learning and memory (B-CALM). B-CALM provides an integrated suite that can control a host of associative learning and memory behaviors. As proof of principle for its applicability, we show data from head-fixed mice learning Pavlovian conditioning, operant conditioning, discrimination learning, as well as a timing task and a choice task. These can be run directly from a user-friendly graphical user interface (GUI) written in MATLAB that controls many independently running Arduino Mega microcontrollers in parallel (one per behavior box). In sum, B-CALM will enable researchers to execute a wide variety of associative learning and memory tasks in a scalable, accurate, and user-friendly manner.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ainslie, G., & Herrnstein, R. J. (1981). Preference reversal and delayed reinforcement. Animal Learning & Behavior, 9, 476–482.
Ainslie, G. W. (1974). Impulse control in pigeons. Journal of the Experimental Analysis of Behavior, 21, 485–489.
Akam, T., Lustig, A., Rowland, J., Kapanaiah, S. K. T., Esteve-Agraz, J., Panniello, M., Marquez, C., Kohl, M., Kätzel, D., Costa, R. M., & Walton, M. (2022). Open source, Python-based, hardware and software for controlling behavioural neuroscience experiments. eLife. 11:e67846
Belsey, P. P., Nicholas, M. A., & Yttri, E. A. (2020). Open-source joystick Manipulandum for decision-making, reaching, and motor control studies in mice. eNeuro, 7.
Brunton, B. W., Botvinick, M. M., & Brody, C. D. (2013). Rats and humans can optimally accumulate evidence for decision-making. Science, 340, 95–98.
Burgess, C. P., Lak, A., Steinmetz, N. A., Zatka-Haas, P., Bai Reddy, C., Jacobs, E. A. K., Linden, J. F., Paton, J. J., Ranson, A., Schröder, S., Soares, S., Wells, M. J., Wool, L. E., Harris, K. D., & Carandini, M. (2017). High-yield methods for accurate two-alternative visual psychophysics in head-fixed mice. Cell Reports, 20, 2513–2524.
Burke, D. A., Jeong, H., Wu, B., Lee, S. A., Floeder, J. R., & Namboodiri, V. M. K. (2023). Few-shot learning: Temporal scaling in behavioral and dopaminergic learning. bioRxiv. https://doi.org/10.1101/2023.03.31.535173
Chung, S.-H., & Herrnstein, R. J. (1967). Choice and delay of reinforcement. Journal of the Experimental Analysis of Behavior, 10, 67–74.
Crouse, R. B., Kim, K., Batchelor, H. M., Girardi, E. M., Kamaletdinova, R., Chan, J., Rajebhosale, P., Pittenger, S. T., Role, L. W., Talmage, D. A., Jing, M., Li, Y., Gao, X.-B., Mineur, Y. S., & Picciotto, M. R. (2020). Acetylcholine is released in the basolateral amygdala in response to predictors of reward and enhances the learning of cue-reward contingency. eLife, 9, e57335.
D’Ausilio, A. (2012). Arduino: A low-cost multipurpose lab equipment. Behavior Research Methods, 44, 305–313.
Elliott, C., Vijayakumar, V., Zink, W., & Hansen, R. (2007). National Instruments LabVIEW: A programming environment for laboratory automation and measurement. JALA. Journal of the Association for Laboratory Automation, 12, 17–24.
Fanselow, M. S., & Wassum, K. M. (2016). The origins and Organization of Vertebrate Pavlovian Conditioning. Cold Spring Harbor Perspectives in Biology, 8, a021717.
Ghazizadeh, A., Ambroggi, F., Odean, N., & Fields, H. L. (2012). Prefrontal cortex mediates extinction of responding by two distinct neural mechanisms in Accumbens Shell. The Journal of Neuroscience, 32, 726–737.
Goldstein, R. Z., & Volkow, N. D. (2002). Drug addiction and its underlying neurobiological basis: Neuroimaging evidence for the involvement of the frontal cortex. The American Journal of Psychiatry, 159, 1642–1652.
Gordon-Fennell, A., Barbakh, J. M., Utley, M., Singh, S., Bazzino, P., Gowrishankar, R., Bruchas, M. R., Roitman, M. F., & Stuber, G. D. (2023). An open-source platform for head-fixed operant and Consummatory behavior. bioRxiv. https://doi.org/10.1101/2023.01.13.523828
Herrnstein, R. J. (1961). Relative and absolute strength of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior, 4, 267–272.
Jaramillo, S., & Zador, A. M. (2011). The auditory cortex mediates the perceptual effects of acoustic temporal expectation. Nature Neuroscience, 14, 246–251.
Jeong, H., Taylor, A., Floeder, J. R., Lohmann, M., Mihalas, S., Wu, B., Zhou, M., Burke, D. A., & Namboodiri, V. M. K. (2022). Mesolimbic dopamine release conveys causal associations. Science, 378, eabq6740.
K Namboodiri VM, Hobbs T, Trujillo-Pisanty I, Simon RC, Gray MM, Stuber GD (2021) Relative salience signaling within a thalamo-orbitofrontal circuit governs learning rate. Current Biology 31:5176-5191.e5.
Namboodiri, K., & Stuber, G. D. (2021). The learning of prospective and retrospective cognitive maps within neural circuits. Neuron. 109, 22, 3552–3575
Kim, K.-U., Huh, N., Jang, Y., Lee, D., & Jung, M. W. (2015). Effects of fictive reward on rat’s choice behavior. Scientific Reports, 5, 8040.
Laguesse, S., Morisot, N., Shin, J. H., Liu, F., Adrover, M. F., Sakhai, S. A., Lopez, M. F., Phamluong, K., Griffin, W. C., Becker, H. C., Bender, K. J., Alvarez, V. A., & Ron, D. (2017). Prosapip1-dependent synaptic adaptations in the nucleus Accumbens drive alcohol intake, seeking, and reward. Neuron, 96, 145–159.e8.
Livneh, Y., Ramesh, R. N., Burgess, C. R., Levandowski, K. M., Madara, J. C., Fenselau, H., Goldey, G. J., Diaz, V. E., Jikomes, N., Resch, J. M., Lowell, B. B., & Andermann, M. L. (2017). Homeostatic circuits selectively gate food cue responses in insular cortex. Nature, 546, 611–616.
MacNiven, K. H., Jensen, E. L. S., Borg, N., Padula, C. B., Humphreys, K., & Knutson, B. (2018). Association of Neural Responses to drug cues with subsequent relapse to stimulant use. JAMA Network Open, 1, e186466.
Matell, M. S., & Portugal, G. S. (2007). Impulsive responding on the peak-interval procedure. Behavioural Processes, 74, 198–208.
Matikainen-Ankney, B. A., et al. (2021). An open-source device for measuring food intake and operant behavior in rodent home-cages. eLife, 10, e66173.
Mohebi, A., Pettibone, J. R., Hamid, A. A., Wong, J.-M. T., Vinson, L. T., Patriarchi, T., Tian, L., Kennedy, R. T., & Berke, J. D. (2019). Dissociable dopamine dynamics for learning and motivation. Nature, 570, 65–70.
Monroe, S. C., & Radke, A. K. (2021). Aversion-resistant fentanyl self-administration in mice. Psychopharmacology (Berl), 238, 699–710.
Namboodiri, V. M. K., Huertas, M. A., Monk, K. J., Shouval, H. Z., & Hussain Shuler, M. G. (2015). Visually cued action timing in the primary visual cortex. Neuron, 86, 319–330.
Namboodiri VMK, Otis JM, Heeswijk K van, Voets ES, Alghorazi RA, Rodriguez-Romaguera J, Mihalas S, Stuber GD (2019) Single-cell activity tracking reveals that orbitofrontal neurons acquire and maintain a long-term memory to guide behavioral adaptation. Nature Neuroscience 22:1110.
Nikbakht, N., & Diamond, M. E. (2021). Conserved visual capacity of rats under red light. eLife, 10, e66429.
Otis, J. M., Namboodiri, V. M. K., Matan, A. M., Voets, E. S., Mohorn, E. P., Kosyk, O., McHenry, J. A., Robinson, J. E., Resendez, S. L., Rossi, M. A., & Stuber, G. D. (2017). Prefrontal cortex output circuits guide reward seeking through divergent cue encoding. Nature, 543, 103–107.
Pavlov, I. P. (1927). Conditioned reflexes: An investigation of the physiological activity of the cerebral cortex, conditioned reflexes: An investigation of the physiological activity of the cerebral cortex. Oxford, England: Oxford Univ. Press.
Perry, C. J., Zbukvic, I., Kim, J. H., & Lawrence, A. J. (2014). Role of cues and contexts on drug-seeking behaviour. British Journal of Pharmacology, 171, 4636–4672.
Poddar, R., Kawai, R., & Ölveczky, B. P. (2013). A fully automated high-throughput training system for rodents. PLoS One, 8, e83171.
Richard, J. M., Ambroggi, F., Janak, P. H., & Fields, H. L. (2016). Ventral pallidum neurons encode incentive value and promote Cue-elicited instrumental actions. Neuron, 90, 1165–1173.
Roberts, S. (1981). Isolation of an internal clock. Journal of Experimental Psychology. Animal Behavior Processes, 7, 242–268.
Saunders, J. L., & Wehr, M. (2019). Autopilot: Automating behavioral experiments with lots of raspberry Pis. bioRxiv. https://doi.org/10.1101/807693
Schultz, B. G., & van Vugt, F. T. (2016). Tap Arduino: An Arduino microcontroller for low-latency auditory feedback in sensorimotor synchronization experiments. Behavior Research Methods, 48, 1591–1607.
Slotnick, B. (2009). A simple 2-transistor touch or lick detector circuit. Journal of the Experimental Analysis of Behavior, 91, 253–255.
Solari, N., Sviatkó, K., Laszlovszky, T., Hegedüs, P., & Hangya, B. (2018). Open source tools for temporally controlled rodent behavior suitable for electrophysiology and Optogenetic manipulations. Frontiers in Systems Neuroscience, 12, 18.
Staddon, J. E. R., & Cerutti, D. T. (2003). Operant conditioning. Annual Review of Psychology, 54, 115–144.
Swanson K, White SR, Preston MW, Wilson J, Mitchell M, Laubach M (2021) An Open Source Platform for Presenting Dynamic Visual Stimuli. eNeuro 8:ENEURO.0563-20.2021.
The International Brain Laboratory et al. (2021). Standardized and reproducible measurement of decision-making in mice. eLife, 10, e63711.
Thorndike, E. L. (1898). Animal intelligence: An experimental study of the associative processes in animals. The Psychological Review: Monograph Supplements, 2, i–109.
Toda, K., Lusk, N. A., Watson, G. D. R., Kim, N., Lu, D., Li, H. E., Meck, W. H., & Yin, H. H. (2017). Nigrotectal stimulation stops interval timing in mice. Current Biology, 27, 3763–3770.e3.
Vollmer, K. M., Doncheck, E. M., Grant, R. I., Winston, K. T., Romanova, E. V., Bowen, C. W., Siegler, P. N., Bobadilla, A.-C., Trujillo-Pisanty, I., Kalivas, P. W., & Otis, J. M. (2020). Drug self-administration in head-restrained mice for simultaneous multiphoton imaging. bioRxiv, 2020.10.25.354209.
White, S. R., Amarante, L. M., Kravitz, A. V., & Laubach, M. (2019). The future is open: Open-source tools for behavioral neuroscience research. eNeuro 6:ENEURO.0223-19.2019.
Woods, N. I., Stefanini, F., Apodaca-Montano, D. L., Tan, I. M. C., Biane, J. S., & Kheirbek, M. A. (2020). The dentate gyrus classifies cortical representations of learned stimuli. Neuron, 107, 173–184.e6.
Yttri, E. A., & Dudman, J. T. (2016). Opponent and bidirectional control of movement velocity in the basal ganglia. Nature, 533, 402–406.
Acknowledgments
We would like to thank Ahmed Morsi and Vincent Lee for help with collecting some behavioral data. We would like to thank Garret Stuber in whose lab V.M.K.N. initially developed the system for controlling Pavlovian conditioning. This work was funded by grants from the National Institutes of Health (R00MH118422, R01MH129582, R01AA029661) and the Scott Alan Myers Endowed Professorship to V.M.K.N. The authors declare no conflict of interest.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhou, M., Wu, B., Jeong, H. et al. An open-source behavior controller for associative learning and memory (B-CALM). Behav Res (2023). https://doi.org/10.3758/s13428-023-02182-6
Accepted:
Published:
DOI: https://doi.org/10.3758/s13428-023-02182-6