Advertisement

Reservoir Computing with a Small-World Network for Discriminating Two Sequential Stimuli

  • Ke Bai
  • Fangzhou Liao
  • Xiaolin HuEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10261)

Abstract

Recently, reservoir network was used for simulating the sequential stimuli discrimination process of monkeys. To deal with the inefficient memory problem of a randomly connected network, a winner-take-all subnetwork was used. In this study, we show that a network with the small-world property makes the WTA subnetwork unnecessary. Using the reinforcement learning in the output layer only, the proposed network successfully learns to accomplish the same discrimination task. In addition, the model neurons exhibit heterogeneous firing properties, which is consistent with the physiological data.

Keywords

Reservoir computing Small-world network Decision making Reinforcement learning 

Notes

Acknowledgments

This work was supported in part by the National Basic Research Program (973 Program) of China under Grant 2013CB329403, in part by the National Natural Science Foundation of China under Grant 91420201, Grant 61332007, and Grant 61621136008, and in part by the German Research Foundation (DFG) under Grant TRR-169.

References

  1. 1.
    Baddeley, A.D.: Working Memory. Oxfoid Psychology Series, vol. 11. Oxford University Press, New York (1986)Google Scholar
  2. 2.
    Barak, O., Sussillo, D., Romo, R., Tsodyks, M., Abbott, L.: From fixed points to chaos: three models of delayed discrimination. Prog. Neurobiol. 103, 214–222 (2013)CrossRefGoogle Scholar
  3. 3.
    Cheng, Z., Deng, Z., Hu, X., Zhang, B., Yang, T.: Efficient reinforcement learning of a reservoir network model of parametric working memory achieved with a cluster population winner-take-all readout mechanism. J. Neurophysiol. 114(6), 3296–3305 (2015)CrossRefGoogle Scholar
  4. 4.
    Draper, N.R., Smith, H.: Applied Regression Analysis. Wiley, Hoboken (2014)zbMATHGoogle Scholar
  5. 5.
    Jaeger, H.: The “echo state” approach to analysing and training recurrent neural networks-with an erratum note. Bonn Ger.: Ger. Natl. Res. Center Inf. Technol. GMD Tech. Rep. 148, 34 (2001)Google Scholar
  6. 6.
    Jun, J.K., Miller, P., Hernández, A., Zainos, A., Lemus, L., Brody, C.D., Romo, R.: Heterogenous population coding of a short-term memory and decision task. J. Neurosci. 30(3), 916–929 (2010)CrossRefGoogle Scholar
  7. 7.
    Loewenstein, Y., Seung, H.S.: Operant matching is a generic outcome of synaptic plasticity based on the covariance between reward and neural activity. Proc. Natl. Acad. Sci. 103(41), 15224–15229 (2006)CrossRefGoogle Scholar
  8. 8.
    Maass, W., Natschläger, T., Markram, H.: Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14(11), 2531–2560 (2002)CrossRefzbMATHGoogle Scholar
  9. 9.
    Machens, C.K., Romo, R., Brody, C.D.: Flexible control of mutual inhibition: a neural model of two-interval discrimination. Science 307(5712), 1121–1124 (2005)CrossRefGoogle Scholar
  10. 10.
    Machens, C.K., Romo, R., Brody, C.D.: Functional, but not anatomical, separation of “what” and “when” in prefrontal cortex. J. Neurosci. 30(1), 350–360 (2010)CrossRefGoogle Scholar
  11. 11.
    Miller, P., Wang, X.J.: Inhibitory control by an integral feedback signal in prefrontal cortex: a model of discrimination between sequential stimuli. Proc. Natl. Acad. Sci. Unit. States Am. 103(1), 201–206 (2006)CrossRefGoogle Scholar
  12. 12.
    Romo, R., Hernández, A., Zainos, A., Lemus, L., Brody, C.D.: Neuronal correlates of decision-making in secondary somatosensory cortex. Nat. Neurosci. 5(11), 1217–1225 (2002)CrossRefGoogle Scholar
  13. 13.
    Romo, R., Salinas, E.: Flutter discrimination: neural codes, perception, memory and decision making. Nat. Rev. Neurosci. 4(3), 203–218 (2003)CrossRefGoogle Scholar
  14. 14.
    Sompolinsky, H., Crisanti, A., Sommers, H.J.: Chaos in random neural networks. Phys. Rev. Lett. 61(3), 259–262 (1988)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Department of PhysicsTsinghua UniversityBeijingChina
  2. 2.Department of Biomedical EngineeringTsinghua UniversityBeijingChina
  3. 3.Department of Computer Science and TechnologyTsinghua National Laboratory for Information Science and TechnologyBeijingChina
  4. 4.Brain-Inspired Computing Research CenterTsinghua UniversityBeijingChina

Personalised recommendations