Advertisement

Spiking neural networks modelled as timed automata: with parameter learning

  • Elisabetta De Maria
  • Cinzia Di GiustoEmail author
  • Laetitia Laversa
Article
  • 13 Downloads

Abstract

In this paper we address the issue of automatically learning parameters of spiking neural networks. Biological neurons are formalized as timed automata and synaptical connections are represented as shared channels among these automata. Such a formalism allows us to take into account several time-related aspects, such as the influence of past inputs in the computation of the potential value of each neuron, or the presence of the refractory period, a lapse of time immediately following the spike emission in which the neuron cannot emit. The proposed model is then formally validated: more precisely, we ensure that some relevant properties expressed as temporal logical formulae hold in the model. Once the validation step is accomplished, we take advantage of the proposed model to write an algorithm for learning synaptical weight values such that an expected behavior can be displayed. The technique we present takes inspiration from supervised learning ones: we compare the effective output of the network to the expected one and backpropagate proper corrective actions in the network. We develop several case studies including a mutual inhibition network.

Keywords

Neural networks Parameter learning Timed automata Temporal logic Model checking 

Notes

Acknowledgements

We are grateful to Giovanni Ciatto for his preliminary implementation work and for his enthusiasm in collaborating with us.

References

  1. Ackley DH, Hinton GE, Sejnowski TJ (1988) A learning algorithm for Boltzmann machines. In: Waltz D, Feldman JA (eds) Connectionist models and their implications: readings from cognitive science. Ablex Publishing Corp., Norwood, pp 285–307Google Scholar
  2. Alur R, Dill DL (1994) A theory of timed automata. Theor Comput Sci 126(2):183–235MathSciNetCrossRefzbMATHGoogle Scholar
  3. Aman B, Ciobanu G (2016) Modelling and verification of weighted spiking neural systems. Theor Comput Sci 623:92–102.  https://doi.org/10.1016/j.tcs.2015.11.005 MathSciNetCrossRefzbMATHGoogle Scholar
  4. Bengtsson J, Larsen KG, Larsson F, Pettersson P, Yi W (1995) Uppaal—a tool suite for automatic verification of real-time systems. In: Proceedings of workshop on verification and control of hybrid systems III, no. 1066 in Lecture notes in computer science. Springer, pp 232–243Google Scholar
  5. Bohte SM, Poutr HAL, Kok JN, La HA, Joost P, Kok N (2002) Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing 48:17–37CrossRefzbMATHGoogle Scholar
  6. Ciatto G, De Maria E, Di Giusto C (2016) Additional material. https://github.com/gciatto/snn_as_ta
  7. Cinquin O, Demongeot J (2005) High-dimensional switches and the modelling of cellular differentiation. J Theor Biol 233(3):391–411CrossRefGoogle Scholar
  8. Clarke EM Jr, Grumberg O, Peled DA (1999) Model checking. MIT Press, CambridgeGoogle Scholar
  9. Cybenko G (1989) Approximation by superpositions of a sigmoidal function. Math Control Signals Syst 2(4):303–314MathSciNetCrossRefzbMATHGoogle Scholar
  10. D’Angelo E, Danese G, Florimbi G, Leporati F, Majani A, Masoli S, Solinas S, Torti E (2015) The human brain project: high performance computing for brain cells hw/sw simulation and understanding. In: 2015 Euromicro conference on digital system design, DSD 2015, Madeira, Portugal, August 26–28, 2015. IEEE Computer Society, pp 740–747.  https://doi.org/10.1109/DSD.2015.80
  11. De Maria E, Di Giusto C (2018) Parameter learning for spiking neural networks modelled as timed automata. In: Anderson P, Gamboa H, Fred ALN, Badia SB (eds) Proceedings of the 11th international joint conference on biomedical engineering systems and technologies (BIOSTEC 2018)—volume 3: BIOINFORMATICS, Funchal, January 19–21, 2018. SciTePress, pp 17–28.  https://doi.org/10.5220/0006530300170028
  12. De Maria E, Muzy A, Gaffé D, Ressouche A, Grammont F (2016) Verification of temporal properties of neuronal archetypes using synchronous models. In: 5th International workshop on hybrid systems biology, GrenobleGoogle Scholar
  13. De Maria E, Di Giusto C, Ciatto G (2017a) Formal validation of neural networks as timed automata. In: Proceedings of the 8th international conference on computational systems-biology and bioinformatics, Nha Trang City, Viet Nam, December 7–8, 2017. ACM, pp 15–22.  https://doi.org/10.1145/3156346.3156350
  14. De Maria E, Di Giusto C, Laversa L (2017b) Additional material. https://digiusto.bitbucket.io/
  15. Ermentrout GB, Kopell N (1986) Parabolic bursting in an excitable system coupled with a slow oscillation. SIAM J Appl Math 46(2):233–253MathSciNetCrossRefzbMATHGoogle Scholar
  16. Freund Y, Schapire RE (1999) Large margin classification using the perceptron algorithm. Mach Learn 37(3):277–296CrossRefzbMATHGoogle Scholar
  17. Gerstner W, Kistler W (2002) Spiking neuron models: an introduction. Cambridge University Press, New YorkCrossRefzbMATHGoogle Scholar
  18. Gruber H, Holzer M, Kiehn A, König B (2005) On timed automata with discrete time-structural and language theoretical characterization. In: Developments in language theory, 9th international conference, DLT 2005, Palermo, Italy, July 4–8, 2005, Proceedings, pp 272–283.  https://doi.org/10.1007/1150587724
  19. Gürning A, Bohte S (2014) Spiking neural networks: principles and challenges. In: 22nd European symposium on artificial neural networks, ESANN 2014, Bruges, Belgium, April 23–25, 2014. http://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2014-13.pdf
  20. Hebb DO (1949) The organization of behavior. Wiley, HobokenGoogle Scholar
  21. Hodgkin AL, Huxley AF (1952) A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol 117(4):500–544CrossRefGoogle Scholar
  22. Hopfield JJ (1988) Neural networks and physical systems with emergent collective computational abilities. In: Anderson JA, Rosenfeld E (eds) Neurocomputing: foundations of research. MIT Press, Cambridge, pp 457–464Google Scholar
  23. Ijspeert AJ (2008) Central pattern generators for locomotion control in animals and robots: a review. Neural Netw 21(4):642–653.  https://doi.org/10.1016/j.neunet.2008.03.014 CrossRefGoogle Scholar
  24. Izhikevich EM (2003) Simple model of spiking neurons. Trans Neural Netw 14(6):1569–1572MathSciNetCrossRefGoogle Scholar
  25. Izhikevich EM (2004) Which model to use for cortical spiking neurons? IEEE Trans Neural Netw 15(5):1063–1070CrossRefGoogle Scholar
  26. Lapicque L (1907) Recherches quantitatives sur l’excitation electrique des nerfs traitee comme une polarization. J Physiol Pathol Gen 9:620–635Google Scholar
  27. Maass W (1997) Networks of spiking neurons: the third generation of neural network models. Neural Netw 10(9):1659–1671CrossRefGoogle Scholar
  28. Matsuoka K (1987) Mechanisms of frequency and pattern control in the neural rhythm generators. Biol Cybern 56(5–6):345–353CrossRefGoogle Scholar
  29. McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5(4):115–133MathSciNetCrossRefzbMATHGoogle Scholar
  30. Paugam-Moisy H, Bohte S (2012) Computing with spiking neuron networks. Springer, Berlin, pp 335–376Google Scholar
  31. Recce M (1999) Encoding information in neuronal activity. In: Maass W, Bishop CM (eds) Pulsed neural networks. MIT Press, Cambridge, pp 111–131Google Scholar
  32. Richard A (2010) Negative circuits and sustained oscillations in asynchronous automata networks. Adv Appl Math 44(4):378–392MathSciNetCrossRefzbMATHGoogle Scholar
  33. Rumelhart DE, Hinton GE, Williams RJ (1988) Learning representations by back-propagating errors. In: Anderson JA, Rosenfeld E (eds) Neurocomputing: foundations of research. MIT Press, Cambridge, pp 696–699Google Scholar
  34. Sjostrom J, Gerstner W (2010) Spike-timing dependent plasticity. Scholarpedia 5(2):1362CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  • Elisabetta De Maria
    • 1
  • Cinzia Di Giusto
    • 1
    Email author
  • Laetitia Laversa
    • 1
  1. 1.Université Côte d’Azur, CNRS, I3SSophia AntipolisFrance

Personalised recommendations