Integrative Probabilistic Evolving Spiking Neural Networks Utilising Quantum Inspired Evolutionary Algorithm: A Computational Framework

  • Nikola Kasabov
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5506)

Abstract

Integrative evolving connectionist systems (iECOS) integrate principles from different levels of information processing in the brain, including cognitive-, neuronal-, genetic- and quantum, in their dynamic interaction over time. The paper introduces a new framework of iECOS called integrative probabilistic evolving spiking neural networks (ipSNN) that incorporate probability learning parameters. ipSNN utilize a quantum inspired evolutionary optimization algorithm to optimize the probability parameters as these algorithms belong to the class of estimation of distribution algorithms (EDA). Both spikes and input features in ipESNN are represented as quantum bits being in a superposition of two states (1 and 0) defined by a probability density function. This representation allows for the state of an entire ipESNN at any time to be represented probabilistically in a quantum bit register and probabilistically optimised until convergence using quantum gate operators and a fitness function. The proposed ipESNN is a promising framework for both engineering applications and brain data modeling as it offers faster and more efficient feature selection and model optimization in a large dimensional space in addition to revealing new knowledge that is not possible to obtain using other models. Further development of ipESNN are the neuro-genetic models – ipESNG, that are introduced too, along with open research questions.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Abbott, L.F., Sacha, B.: Synaptic plasticity: taming the beast. Nature Neuroscience 3, 1178–1183 (2000)CrossRefGoogle Scholar
  2. 2.
    Ackley, D.H., Hinton, G.E., Sejnowski, T.J.: A learning algorithm for Boltzmann machines. Cognitive Science 9, 147–169 (1985)CrossRefGoogle Scholar
  3. 3.
    Arbib, M. (ed.): The Handbook of Brain Theory and Neural Networks. MIT Press, Cambridge (2003)MATHGoogle Scholar
  4. 4.
    Belatreche, A., Maguire, L.P., McGinnity, M.: Advances in Design and Application of Spiking Neural Networks. Soft Comput. 11(3), 239–248 (2006)CrossRefMATHGoogle Scholar
  5. 5.
    Benuskova, L., Kasabov, N.: Comput. Neurogenetic Modelling. Springer, NY (2007)CrossRefGoogle Scholar
  6. 6.
    Bershadskii, A., et al.: Brain neurons as quantum computers: in vivo support of background physics. Reports of the Bar-Ilan University, Israel, vol. 1-12 (2003)Google Scholar
  7. 7.
    Brette, R., et al.: Simulation of networks of spiking neurons: A review of tools and strategies. Journal of Computational Neuroscience 23(3), 349–398 (2007)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Castellani, M.: ANNE - A New Algorithm for Evolution of ANN Classifier Systems. In: IEEE Congress on Evolutionary Computation, CEC 2006, pp. 3294–3301 (2006)Google Scholar
  9. 9.
    Dayan, P., Hinton, G.E.: Varieties of Helmholtz machines. Neural Networks 9, 1385–1403 (1996)CrossRefMATHGoogle Scholar
  10. 10.
    Dayan, P., Hinton, G.E., Neal, R., Zemel, R.S.: The Helmholtz machine. Neural Computation 7, 1022–1037 (1995)Google Scholar
  11. 11.
    Defoin-Platel, M., Schliebs, S., Kasabov, N.: Quantum-inspired Evolutionary Algorithm: A multi-model EDA. IEEE Trans. Evolutionary Computation (in print, 2009)Google Scholar
  12. 12.
    Deutsch, D.: Quantum computational networks. Proceedings of the Royal Society of London A(425), 73–90 (1989)Google Scholar
  13. 13.
    Ezhov, A., Ventura, D.: Quantum neural networks, in Future Directions for Intelligent Systems and Information Sciences. In: Kasabov, N. (ed.) Future directions for intelligent systems. Springer, Heidelberg (2000)Google Scholar
  14. 14.
    Gerstner, W., Kistler, W.M.: Spiking Neuron Models. Cambridge Univ. Press, Cambridge (2002)CrossRefMATHGoogle Scholar
  15. 15.
    Gerstner, W.: What’s different with spiking neurons? In: Mastebroek, H., Vos, H. (eds.) Plausible Neural Networks for Biological Modelling, pp. 23–48. Kl. Ac. Publ., Dordrecht (2001)CrossRefGoogle Scholar
  16. 16.
    Guyon, I., et al. (eds.): Feature Extraction, Foundations and Applications. Springer, Heidelberg (2006)Google Scholar
  17. 17.
    Han, K.-H., Kim, J.-H.: Quantum-inspired evolutionary algorithm for a class of combinatorial optimization. IEEE Trans. on Evolutionary Computation, 580–593 (2005)Google Scholar
  18. 18.
    Hey, T.: Quantum computing: an introduction. Comp. & Control Eng. J. 10(6) (1999)Google Scholar
  19. 19.
    Hinton, G.E., Dayan, P., Frey, B.J., Neal, R.: The wake-sleep algorithm for unsupervised neural networks. Science 268, 1158–1161 (1995)CrossRefGoogle Scholar
  20. 20.
    Hirvensalo, M.: Quantum computing. Springer, Heidelberg (2004)CrossRefMATHGoogle Scholar
  21. 21.
    Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Huguenard, J.R.: Reliability of axonal propagation: The spike doesn’t stop here. PNAS 97(17), 9349–9350 (2000)CrossRefGoogle Scholar
  23. 23.
    Izhikevich, E., Desai, N.: Relating STDP to BCM. Neural Comp. 15, 1511–1523 (2003)CrossRefMATHGoogle Scholar
  24. 24.
    Izhikevich, E.: Simple model of spiking neurons. IEEE Tr. NN 14(6), 1569–1572 (2003)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Izhikevich, E.: Which model to use for cortical spiking neurons? IEEE Tr. NN 15(5), 1063–1070 (2004)CrossRefGoogle Scholar
  26. 26.
    Kasabov, N.: Evolving Connectionist Systems: The Knowl. Eng. Appr. Springer, Heidelberg (2007)Google Scholar
  27. 27.
    Kasabov, N.: Integrative Connectionist Learning Systems Inspired by Nature: Current Models, Future Trends and Challenges. Natural Computing. Springer, Heidelberg (2008)MATHGoogle Scholar
  28. 28.
    Kasabov, N.: Brain-, Gene-, and Quantum Inspired Computational Intelligence: Challenges and Opportunities. In: Duch, W., Manzduk, J. (eds.) Challenges in Computational Intelligence, pp. 193–219. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  29. 29.
    Kasabov, N.: Evolving Intelligence in Humans and Machines: Integrative Evolving Connectionist Systems Approach. IEEE Computational Intelligence Magazine 3(3), 23–37 (2008)CrossRefGoogle Scholar
  30. 30.
    Kasabov, N.: Found. of neural networks, fuzzy systems and knowl. eng. MIT Press, Cambridge (1996)Google Scholar
  31. 31.
    Katsumata, S., Sakai, K., Toujoh, S., Miyamoto, A., Nakai, J., Tsukada, M., Kojima, H.: Analysis of synaptic transmission and its plasticity by glutamate receptor channel kinetics models and 2-photon laser photolysis. In: Proc. of ICONIP 2008. LNCS. Springer, Heidelberg (2009)Google Scholar
  32. 32.
    Kohonen, T.: Self-Organizing Maps, 2nd edn. Springer, Heidelberg (1997)CrossRefMATHGoogle Scholar
  33. 33.
    Kistler, G., Gerstner, W.: Spiking Neuron Models - Single Neurons, Populations, Plasticity. Cambridge Univ. Press, Cambridge (2002)MATHGoogle Scholar
  34. 34.
    Maass, W., Bishop, C. (eds.): Pulsed Neural Networks. MIT Press, Cambridge (1999)MATHGoogle Scholar
  35. 35.
    Pavlidis, N.G., Tasoulis, O.K., Plagianakos, V.P., Nikiforidis, G., Vrahatis, M.N.: Spiking neural network training using evolutionary algorithms. In: Proceedings IEEE International Joint Conference on Neural Networks, vol. 4, pp. 2190–2194 (2005)Google Scholar
  36. 36.
    Pfister, J.P., Barber, D., Gerstner, W.: Optimal Hebbian Learning: a Probabilistic Point of View. In: Kaynak, O., Alpaydın, E., Oja, E., Xu, L. (eds.) ICANN 2003 and ICONIP 2003. LNCS, vol. 2714, pp. 92–98. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  37. 37.
    Bohte, S.M., La Poutré, H.A., Kok, J.N.: Error-Backpropagation in Temporally Encoded Networks of Spiking Neurons. Neurocomputing 48(1-4), 17–37 (2002)CrossRefMATHGoogle Scholar
  38. 38.
    Bohte, S.M., Kok, J.N.: Applications of spiking neural networks. Information Processing Letters 95(6), 519–520 (2005)CrossRefMATHGoogle Scholar
  39. 39.
    Schliebs, S., Defoin-Platel, M., Kasabov, N.: Integrated Feature and Parameter Optimization for an Evolving Spiking Neural Network. In: Proc. of ICONIP 2008, Auckland, NZ. LNCS, vol. 5506/5507. Springer, Heidelberg (2009)Google Scholar
  40. 40.
    Soltic, W.S., Kasabov, N.: Evolving spiking neural networks for taste recognition. In: Proc. WCCI 2008, Hong Kong. IEEE Press, Los Alamitos (2008)Google Scholar
  41. 41.
    Specht, D.F.: Enhancements to probabilistic neural networks. In: Proc. Int. Joint Conference on Neural Networks, June 1992, vol. 1, pp. 761–768 (1992)Google Scholar
  42. 42.
    Tuffy, F., McDaid, L., Wong Kwan, V., Alderman, J., McGinnity, T.M., Kelly, P., Santos, J.: Spiking Neuron Cell Based on Charge Coupled Synapses. In: Proc. IJCNN, Vancouver (2006)Google Scholar
  43. 43.
    Ventura, D., Martinez, T.: Quantum associative memory. Information Sciences 124(1-4), 273–296 (2000)MathSciNetCrossRefGoogle Scholar
  44. 44.
    Verstraeten, D., Schrauwen, B., Stroobandt, D., Van Campenhout, J.: Isolated word recog. with the Liquid State Machine: a case study. Inf. Proc. Letters 95(6), 521–528 (2005)CrossRefMATHGoogle Scholar
  45. 45.
    Villa, A.E.P., et al.: Cross-channel coupling of neuronal activity in parvalbumin-deficient mice susceptible to epileptic seizures. Epilepsia 46(suppl. 6), 359 (2005)Google Scholar
  46. 46.
    Wysoski, S., Benuskova, L., Kasabov, N.: On-line learning with structural adaptation in a network of spiking neurons for visual pattern recognition. In: Kollias, S.D., Stafylopatis, A., Duch, W., Oja, E. (eds.) ICANN 2006. LNCS, vol. 4131, pp. 61–70. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  47. 47.
    Wysoski, S., Benuskova, L., Kasabov, N.: Brain-like Evolving Spiking Neural Networks for Multimodal Information. In: Proc. ICONIP 2007, Kitakyushu. LNCS. Springer, Heidelberg (2007)Google Scholar
  48. 48.
    Yadav, A., Mishra, D., Yadav, R.N., Ray, S., Kalra, P.K.: Time-series prediction with single integrate-and-fire neuron. Applied Soft Computing 7(3), 739–745 (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Nikola Kasabov
    • 1
  1. 1.Knowledge Engineering and Discovery Research Institute, KEDRIAuckland University of TechnologyAucklandNew Zealand

Personalised recommendations