Advertisement

An Oscillatory Neural Network Model for Birdsong Learning and Generation: Implications for the Role of Dopamine in Song Learning

  • M. Maya
  • V. Srinivasa Chakravarthy
  • B. Ravindran
Chapter
Part of the Cognitive Science and Technology book series (CSAT)

Abstract

We present a model of bird song learning and production in which the motor control pathway is modeled by a trainable network of oscillators and the Anterior Forebrain Pathway (AFP) is modeled as a stochastic system. Song learning in many species of birds is divided into two phases. In the first phase, the sensory phase, the male bird listens to the tutor song of another male bird in the colony and memorizes some aspect of the tutor song. In the second phase, the motor learning phase, the bird establishes the songs learnt earlier by rehearsal aided by auditory self-feedback. We hypothesize that: (1) the songbird learns only evaluations of songs during the sensory phase; (2) the AFP plays a role analogous to the Explorer, a key component in reinforcement learning (RL); (3) the motor pathway learns the song by combining the evaluations (value information) stored from the sensory phase, and the exploratory inputs from the AFP in a temporal stage-wise manner. Model performance on real birdsong samples is presented. Impaired song output under conditions of lesions of AFP nuclei, including the Lateral Magnocellular Nucleus of the Anterior Neostriatum (LMAN) and Area X, is studied. The model also proposes a role for dopamine signal in song learning and shows that under dopamine-deficient conditions, similar to those of Parkinson’s disease, song learning is impaired.

Keywords

Birdsong Reinforcement learning Chunking Actor–Critic–Explorer schema Central pattern generators Anterior Forebrain Pathway (AFP) 

References

  1. Abarnel, H. D. I., Gibb, L., Mindlin, G. B., & Talathi, S. (2004). Mapping neural architectures onto acoustic features of birdsong. Journal of Neuroscience, 92(1), 96–110.Google Scholar
  2. Appeltants, D., Absil, P., Balthazart, J., & Ball, G. F. (2000). Identification of the origin of catecholaminergic inputs to HVc in canaries by retrograde tract tracing combined with tyrosine hydroxylase immunocytochemistry. Journal of Chemical Neuroanatomy, 18, 117–133.CrossRefGoogle Scholar
  3. Appletants, D., Ball, G. F., & Balthazart, J. (2002). The origin of catecholaminergic inputs to the song control nucleus RA in canaries. NeuroReport, 13, 649–653.CrossRefGoogle Scholar
  4. Balasubramani, P. P., Chakravarthy, V. S., Ali, M., Ravindran, B., & Moustafa, A. A. (2015). Identifying the basal ganglia network model markers for medication-induced impulsivity in Parkinson’s Disease patients. PLoS ONE, 10, e0127542.CrossRefGoogle Scholar
  5. Barto, A. G. (1995). Adaptive Critics and the Basal Ganglia. In J. C. Houk, J. Davis, & D. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 215–232). Cambridge, MA: MIT Press.Google Scholar
  6. Basham, M. E., Nordeen, E. J., & Nordeen, K. W. (1996). Blockade of NMDA receptors in the anterior forebrain impairs sensory acquisition in the zebra finch (Poephila guttata). Neurobiology of Learning and Memory, 66(3), 295–304.Google Scholar
  7. Bergman, H., Wichmann, T., Karmon, B., & DeLong, M. R. (1994). The primate subthalamic nucleus. II. Neuronal activity in the MPTP model of parkinsonism. Journal of Neurophysiology, 72, 507–520.CrossRefGoogle Scholar
  8. Berns, G. S., & Sejnowski, T. J. (1995). A computational model of local memory in the primate pallidal-subthalamic circuit. Soc. Neurosc. Abstracts, 21, 678.Google Scholar
  9. Berns, G. S., & Sejnowski, T. J. (1998). A computational model of how the Basal ganglia produce sequences. Journal of Cognitive Neuroscience, 10, 108–121.CrossRefGoogle Scholar
  10. Bolhuis, J. J., & Moorman, S. (2015). Birdsong memory and the brain: In search of the template. Neuroscience and Biobehavioral Reviews, 50C, 41–55.CrossRefGoogle Scholar
  11. Bottjer, S. W. (1993). The distribution of tyrosine hydroxylase immunoreactivity in the brains of male and female zebra finches. Journal of Neurobiology, 24, 51–69.CrossRefGoogle Scholar
  12. Bottjer, S. W., Meisner, E. A., & Arnold, A. P. (1984). Forebrain lesions disrupt development but not maintenance of song in passerine birds. Science, 224, 901–903.CrossRefGoogle Scholar
  13. Brainard, M. S., & Doupe, A. J. (2002). What songbirds teach us about learning. Nature, 417, 351–358.CrossRefGoogle Scholar
  14. Brown, P., Olivero, A., Mazzone, P., Insola, A., Tonali, P., & Lazzaro, V. D. (2001). Dopamine dependency of oscillations in between subthalamic nucleus and pallidum in Parkinson’s disease. Journal of Neuroscience, 21, 1033–1038.Google Scholar
  15. Carrilo, G. D., & Doupe, A. J. (2004). Is the songbird Area X striatal, pallidal, or both? An anatomical study. The Journal of Comparative Neurology, 473, 415–437.Google Scholar
  16. Chakravarthy, V. S., & Balasubramani, P. P. (2014). Basal ganglia system as an engine for exploration. In R. Jung & D. Jaeger (Eds.), Encyclopedia of computational neuroscience. Berlin, Heidelberg: Springer.Google Scholar
  17. Chakravarthy, V. S., Joseph, D., & Bapi, R. S. (2010). What do the Basal Ganglia Do? A modeling perspective. Biological Cybernetics.Google Scholar
  18. Cossette, M., Lévesque, M., & Parent, A. (1999). Extrastriatal dopaminergic innervation of human basal ganglia. Neuroscience Research, 34, 51–54.CrossRefGoogle Scholar
  19. Crook, S., & Cohen, A. (2003). Central pattern generators. The book of genesis (Internet edition).Google Scholar
  20. Dave, A. S., & Margoliash, D. (2000). Song replay during sleep and computational rules for sensorimotor vocal learning. Science, 290, 812–816.CrossRefGoogle Scholar
  21. Ding, L., & Perkel, D. J. (2002). Dopamine modulates excitability of spiny neurons in the avian basal ganglia. Journal of Neuroscience, 22, 5210–5218.Google Scholar
  22. Doupe, A. J. (1997). Song –and order – selective neurons in the songbird anterior forebrain and their emergence during vocal development. Jounal of Neuroscience, 17(3), 1147–1167.Google Scholar
  23. Doupe, J. A., & Kuhl, K. P. (1999). Birdsong and human speech: Common themes and mechanisms. Annual Review of Neuroscience, 22, 567–631.CrossRefGoogle Scholar
  24. Doupe, A. J., Perkel, D. J., Reiner, A., & Stern, E. A. (2005). Birdbrains could teach basal ganglia research a new song. Trends in Neurosciences, 28, 353–363.CrossRefGoogle Scholar
  25. Doya, K., & Sejnowski, J. T. (1995). A novel reinforcement model of birdsong vocalization learning. Advances in Neural Information Processing Systems, 7, 101–108.Google Scholar
  26. Doya, K., & Sejnowski, T. J. (1998). A computational model of birdsong learning by auditory experience and auditory feedback. In Central auditory processing and neural modeling (pp. 77-88). Springer, Boston, MA.Google Scholar
  27. Farries, M. A., & Perkel, D. J. (2002). A telencephalic nucleus essential for song learning contains neurons with physiological characteristics of both striatum and globus pallidus. Jounal of Neuroscience, 22, 3776–3787.Google Scholar
  28. Fiete, I. R., Fee, M. S., & Seung, H. S. (2007). Model of birdsong learning based on gradient estimation by dynamic perturbation of neural conductances. Journal of Neurophysiology, 98, 2038–2057.CrossRefGoogle Scholar
  29. Gadagkar, V., Puzerey, P. A., Chen, R., Baird-Daniel, E., Farhang, A. R., & Goldberg, J. H. (2016). Dopamine neurons encode performance error in singing birds. Science, 354(6317), 1278–1282.CrossRefGoogle Scholar
  30. Gale, S. D., & Perkel, D. J. (2005). Properties of dopamine release and uptake in the songbird basal ganglia. Journal of Neurophysiology, 93, 1871–1879.CrossRefGoogle Scholar
  31. Gardner, T., Cecchi, G., Magnasco, M., Laje, R., & Mindlin, G. B. (2001). Simple motor gestures for birdsongs. Physical Review Letters, 87, 208101.CrossRefGoogle Scholar
  32. Hahnloser, R. H., & Kotowicz, A. (2010). Auditory representations and memory in birdsong learning. Current Opinion in Neurobiology, 20, 332–339.CrossRefGoogle Scholar
  33. Hara, E., Kubikova, L., Hessler, N. A., & Jarvis, E. D. (2007). Role of the midbrain dopaminergic system in modulation of vocal brain activation by social context. European Journal of Neuroscience, 25, 3406–3416.CrossRefGoogle Scholar
  34. Harding, C. F. (1998). Changes in catecholamine levels and turnover rates in hypothalamic, vocal control and auditory nuclei in male zebra finches during development. Journal of Neurobiology, 34, 329–346.CrossRefGoogle Scholar
  35. Harel, B. T., Cannizzaro, M. S., Cohen, H., Reilly, N., & Snyder, P. J. (2004). Acoustic characteristics of Parkinsonian speech: A potential biomarker of early disease progression and treatment. Journal of Neurolinguistics, 17, 439–453.CrossRefGoogle Scholar
  36. Hessler, N. A., & Doupe, A. J. (1999). Singing-related neural activity in a dorsal forebrain-basal ganglia circuit of adult zebra finches. Journal of Neuroscience, 19, 10461–10481.Google Scholar
  37. Hoese, W., Podos J, Boetticher, N. C., & Nowicki, S. (2000, June). Vocal tract function in birdsong production: Experimental manipulation of beak movements. Journal of Experimental Biology, 203(12), 1845–1855.Google Scholar
  38. Iyengar, S., & Bottjer, S. J. (2002). Development of individual axon arbors in a thalamocortical circuit necessary for song learning in zebra finches. Journal of Neuroscience, 22(3), 901–911.Google Scholar
  39. Joseph, D., Gangadhar, G., Chakravarthy, V. S. ACE (Actor—Critic—Explorer) paradigm for reinforcement learning in basal ganglia: Highlighting the role of sub-thalamic and pallidal nuclei. Neurocomputing, (2010, in press).Google Scholar
  40. Kao, M. H., Doupe, A. J., & Brainard, M. S. (2005). Contributions of an avian basal ganglia-forebrain circuit to real-time modulation of song. Nature, 433, 638–643.CrossRefGoogle Scholar
  41. Konishi, M. (1965). The role of auditory feedback in the control of vocalization in the white-crowned sparrow. Zeitschrift fur Tierpsychologie, 22, 770–783.Google Scholar
  42. Kubikova, L., & Kostal, L. (2009). Dopaminergic system in birdsong learning and maintenance. Journal of Chemical Neuroanatomy.Google Scholar
  43. Kuhl, P. K. (1994). Learning and representation in speech and language. Current Opinion in Neurobiology, 4, 812–822.Google Scholar
  44. Leonardo, A. (2004). Experimental test of the birdsong error-correction model. Proceedings of the National Academy of Sciences of the United States of America, 101, 16935–16940.CrossRefGoogle Scholar
  45. Lewis, J. W., Ryan, S. M., Arnold, A. P., & Butcher, L. L. (1981). Evidence for a catecholarninergic projection to area X in the zebra finch. Journal of Comparative Neurology, 196, 347–354.CrossRefGoogle Scholar
  46. Luo, M., Ding, L., & Perkel, D. J. (2001). An avian basal ganglia pathway essential for vocal learning forms a closed topographic loop. Journal of Neuroscience, 21, 6836–6845.Google Scholar
  47. Magdoom, K. N., Subramanian, D., Chakravarthy, V. S., Amari, S.-I., & Meenakshisundaram, N. Modelling basal ganglia for understanding Parkinsonian reaching movements. Neural Computation, (2010, in press).Google Scholar
  48. Magnin, M., Morel, A., & Jeanmonod, D. (2000). Single-unit analysis of the pallidum, thalamus and subthalamic nucleus in Parkinsonian patients. Neuroscience, 96, 549–564.CrossRefGoogle Scholar
  49. Margoliash, D. (2002). Evaluating theories of bird song learning: Implications for future directions. Journal of Comparative Physiology. A, Sensory, Neural, and Behavioral Physiology, 188, 851–866.CrossRefGoogle Scholar
  50. Marler, P., & Peters, S. (1977). Selective vocal learning in a sparrow. Science, 198, 519–521.CrossRefGoogle Scholar
  51. McCasland, J. S. (1987). Neuronal control of bird song production. Jounal of Neuroscience,7, 23–39.Google Scholar
  52. Nini A., Feingold, A., Slovin, H., & Bergman, H. (1995). Neurons in the globus pallidus do not show correlated activity in the normal monkey, but phase-locked oscillations appear in the MPTP model of Parkinsonism. Journal of Neurophysiology, 74, 1800–1805.Google Scholar
  53. Nottebohm, F., Stokes, T. M., & Leonard, C. M. (1976). Central control of song in the canary, Serinus canarinus. Journal of Comparative Neurology, 165, 457–486.CrossRefGoogle Scholar
  54. Reiner, A., Perkel, D. J., Mello, C. V., & Jarvis, E. D. (2004). Songbirds and the revised avian brain nomenclature. Annals of the New York Academy of Sciences, 1016, 77–108.CrossRefGoogle Scholar
  55. Righetti, L., Buchli, J., & Ijspeert, A. J. (2006). Dynamic Hebbian learning in adaptive frequency oscillators. Physica D: Nonlinear Phenomena, 216, 269–281.MathSciNetCrossRefzbMATHGoogle Scholar
  56. Sakai, K., Kitaguchi, K., & Hikosaka, O. (2003). Chunking during human visuomotor sequence learning. Experimental Brain Research, 152, 229–242.CrossRefGoogle Scholar
  57. Scharff, C., & Nottebohm, F. (1991). A comparative study of the behavioral deficits following lesions of various parts of the zebra finch song system: Implications for vocal learning. Journal of Neuroscience, 11, 2896–2913.Google Scholar
  58. Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80, 1–27.CrossRefGoogle Scholar
  59. Soha, J. A., Shimizu, T., & Doupe, A. J. (1996). Development of the catecholaminergic innervation of the song system of the male zebra finch. Journal of Neurobiology, 29, 473–489 (464951).Google Scholar
  60. Sohrabji, F., Nordeen, E. J., & Nordeen, K. W. (1990). Selective impairment of song learning following lesions of a forebrain nucleus in the juvenile zebra finch. Behavioral and Neural Biology., 53, 51–63.CrossRefGoogle Scholar
  61. Sridharan, D., Prashanth, P. S., & Chakravarthy, V. S. (2004). The role of the basal ganglia in exploratory behavior in a model based on reinforcement learning. In N. R. Pal, N. Kasabov, R. K. Mudi, S. Pal, & S. K. Parui (Eds.), International Conference on Neural Information Processing, Lecture Notes in Computer Science (LNCS) (Vol. 3316, pp. 70–77). Berlin: Springer.Google Scholar
  62. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.Google Scholar
  63. Trevisan, M. A., Eguia, M. C., & Mindlin, G. B. (2001). Non-linear aspects of analysis and synthesis of analysis and synthesis of speech time series data. Physical Review E, 63, 026216.CrossRefGoogle Scholar
  64. Troyer, T. W., & Doupe, A. J. (2000). An associational model of birdsong sensorimotor learning: Efference copy and the learning of song syllables. Journal of Neurophysiology, 84, 1024–1223.Google Scholar
  65. Vu, E. T., Mazutek, M. E., & Kuo, Y.-C. (1994). Identification of a forebrain motor programming network for the learned song of zebra finches. Journal of Neuroscience, 14, 6924–6934.Google Scholar
  66. Yanagihara, S., & Yazaki-Sugiyama, Y. (2016, June 21). Auditory experience-dependent cortical circuit shaping for memory formation in bird song learning. Nature Communications.Google Scholar
  67. Yu, A. C., & Margoliash, D. (1996). Temporal hierarchical control of singing in birds. Science, 273, 1871–1875.Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  • M. Maya
    • 1
  • V. Srinivasa Chakravarthy
    • 1
  • B. Ravindran
    • 2
  1. 1.Department of BiotechnologyIndian Institute of Technology MadrasChennaiIndia
  2. 2.Department of Computer Science and EngineeringIndian Institute of Technology MadrasChennaiIndia

Personalised recommendations