Advertisement

Third Generation Neural Networks: Spiking Neural Networks

  • Samanwoy Ghosh-Dastidar
  • Hojjat Adeli
Conference paper
Part of the Advances in Intelligent and Soft Computing book series (AINSC, volume 116)

Abstract

Artificial Neural Networks (ANNs) are based on highly simplified brain dynamics and have been used as powerful computational tools to solve complex pattern recognition, function estimation, and classification problems. Throughout their development, ANNs have been evolving towards more powerful and more biologically realistic models. In the last decade, the third generation Spiking Neural Networks (SNNs) have been developed which comprise of spiking neurons. Information transfer in these neurons models the information transfer in biological neurons, i.e., via the precise timing of spikes or a sequence of spikes. Addition of the temporal dimension for information encoding in SNNs yields new insight into the dynamics of the human brain and has the potential to result in compact representations of large neural networks. As such, SNNs have great potential for solving complicated time-dependent pattern recognition problems defined by time series because of their inherent dynamic representation. This article presents an overview of the development of spiking neurons and SNNs within the context of feedforward networks, and provides insight into their potential for becoming the next generation neural networks.

Keywords

Spike Train Radial Basis Function Neural Network Postsynaptic Neuron Presynaptic Neuron Seizure Detection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Adeli, H., Hung, S.L.: Machine Learning - Neural Networks, Genetic Algorithms, and Fuzzy Sets. John Wiley and Sons, New York (1995)Google Scholar
  2. 2.
    Adeli, H., Park, H.S.: Neurocomputing for Design Automation. CRC Press, Boca Raton (1998)Google Scholar
  3. 3.
    Adeli, H., Karim, A.: Fuzzy-wavelet RBFNN model for freeway incident detection. Journal of Transportation Engineering 126(6), 464–471 (2000)CrossRefGoogle Scholar
  4. 4.
    Adeli, H.: Neural networks in civil engineering: 1989-2000. Computer-Aided Civil and Infrastructure Engineering 16(2), 126–142 (2001)CrossRefGoogle Scholar
  5. 5.
    Ghosh-Dastidar, S., Adeli, H.: Wavelet-clustering-neural network model for freeway incident detection. Computer-Aided Civil and Infrastructure Engineering 18(5), 325–338 (2003)CrossRefGoogle Scholar
  6. 6.
    Adeli, H., Karim, A.: Wavelets in Intelligent Transportation Systems. John Wiley and Sons, Hoboken (2005)zbMATHGoogle Scholar
  7. 7.
    Adeli, H., Ghosh-Dastidar, S., Dadmehr, N.: Alzheimer’s disease and models of computation: Imaging, classification, and neural models. Journal of Alzheimer’s Disease 7(3), 187–199 (2005a)Google Scholar
  8. 8.
    Adeli, H., Ghosh-Dastidar, S., Dadmehr, N.: Alzheimer’s disease: Models of computation and analysis of EEGs. Clinical EEG and Neuroscience 36(3), 131–140 (2005b)Google Scholar
  9. 9.
    Adeli, H., Jiang, X.: Dynamic fuzzy wavelet neural network model for structural system identification. Journal of Structural Engineering 132(1), 102–111 (2006)CrossRefGoogle Scholar
  10. 10.
    Ghosh-Dastidar, S., Adeli, H.: Neural network-wavelet microsimulation model for delay and queue length estimation at freeway work zones. Journal of Transportation Engineering 132(4), 331–341 (2006)CrossRefGoogle Scholar
  11. 11.
    Ghosh-Dastidar, S., Adeli, H.: Improved spiking neural networks for EEG classification and epilepsy and seizure detection. Integrated Computer-Aided Engineering 14(3), 187–212 (2007)Google Scholar
  12. 12.
    Adeli, H., Hung, S.L.: A concurrent adaptive conjugate gradient learning algorithm on MIMD machines. Journal of Supercomputer Applications 7(2), 155–166 (1993)CrossRefGoogle Scholar
  13. 13.
    Ghosh-Dastidar, S., Adeli, H., Dadmehr, N.: Mixed-band wavelet-chaos-neural network methodology for epilepsy and epileptic seizure detection. IEEE Transactions on Biomedical Engineering 54(9), 1545–1551 (2007)CrossRefGoogle Scholar
  14. 14.
    Adeli, H., Park, H.S.: Counter propagation neural network in structural engineering. Journal of Structural Engineering 121(8), 1205–1212 (1995a)CrossRefGoogle Scholar
  15. 15.
    Panakkat, A., Adeli, H.: Recurrent neural network for approximate earthquake time and location prediction using multiple seismicity indicators. Computer-Aided Civil and Infrastructure Engineering 24(4), 280–292 (2009)CrossRefGoogle Scholar
  16. 16.
    Karim, A., Adeli, H.: Comparison of the fuzzy-wavelet RBFNN freeway incident detection model with the california algorithm. Journal of Transportation Engineering 128(1), 21–30 (2002)CrossRefGoogle Scholar
  17. 17.
    Karim, A., Adeli, H.: Radial basis function neural network for work zone capacity and queue estimation. Journal of Transportation Engineering 129(5), 494–503 (2003)CrossRefGoogle Scholar
  18. 18.
    Ghosh-Dastidar, S., Adeli, H., Dadmehr, N.: Principal component analysis-enhanced cosine radial basis function neural network for robust epilepsy and seizure detection. IEEE Transactions on Biomedical Engineering 55(2), 512–518 (2008)CrossRefGoogle Scholar
  19. 19.
    Jiang, X., Adeli, H.: Dynamic wavelet neural network model for traffic flow forecasting. Journal of Transportation Engineering 131(10), 771–779 (2005)CrossRefGoogle Scholar
  20. 20.
    Jiang, X., Adeli, H.: Pseudospectra, MUSIC, and dynamic wavelet neural network for damage detection of highrise buildings. International Journal for Numerical Methods in Engineering 71(5), 606–629 (2007)CrossRefGoogle Scholar
  21. 21.
    Jiang, X., Adeli, H.: Dynamic fuzzy wavelet neuroemulator for nonlinear control of irregular highrise building structures. International Journal for Numerical Methods in Engineering 74(7), 1045–1066 (2008)zbMATHCrossRefGoogle Scholar
  22. 22.
    Adeli, H., Hung, S.L.: An adaptive conjugate gradient learning algorithm for effective training of multilayer neural networks. Applied Mathematics and Computation 62(1), 81–102 (1994)zbMATHCrossRefGoogle Scholar
  23. 23.
    Adeli, H., Park, H.S.: Optimization of space structures by neural dynamics. Neural Networks, Vol 8(5), 769–781 (1995b)CrossRefGoogle Scholar
  24. 24.
    Adeli, H., Karim, A.: Neural dynamics model for optimization of cold-formed steel beams. Journal of Structural Engineering 123(11), 1535–1543 (1997)CrossRefGoogle Scholar
  25. 25.
    Adeli, H., Jiang, X.: Neuro-fuzzy logic model for freeway work zone capacity estimation. Journal of Transportation Engineering 129(5), 484–493 (2003)CrossRefGoogle Scholar
  26. 26.
    Sejnowski, T.J.: Open questions about computation in the cerebral cortex. In: Rumelhart, D.E., McClelland, J.L. (eds.) Parallel Distributed Processing, vol. 2, pp. 372–389. MIT Press, Cambridge (1986)Google Scholar
  27. 27.
    Maass, W.: Lower bounds for the computational power of spiking neural networks. Neural Computation 8(1), 1–40 (1996)zbMATHCrossRefMathSciNetGoogle Scholar
  28. 28.
    Maass, W.: Networks of spiking neurons: The third generation of spiking neural network models. Neural Networks 10(9), 1659–1671 (1997a)CrossRefGoogle Scholar
  29. 29.
    Bose, N.K., Liang, P.: Neural Network Fundamentals with Graphs, Algorithms, and Applications. McGraw-Hill, New York (1996)Google Scholar
  30. 30.
    Maass, W.: Noisy spiking neurons with temporal coding have more computational power than sigmoidal neurons. In: Mozer, M., Jordan, M.I., Petsche, T. (eds.) Advances in Neural Information Processing Systems, vol. 9, pp. 211–217. MIT Press, Cambridge (1997b)Google Scholar
  31. 31.
    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. In: Rumelhart, D.E., McClelland, J.L. (eds.) Parallel Distributed Processing, vol. 1, pp. 318–362. MIT Press, Cambridge (1986)Google Scholar
  32. 32.
    Hung, S.L., Adeli, H.: Parallel backpropagation learning algorithms on Cray Y-MP8/864 supercomputer. Neurocomputing 5(6), 287–302 (1993)CrossRefGoogle Scholar
  33. 33.
    Park, H.S., Adeli, H.: Distributed neural dynamics algorithms for optimization of large steel structures. Journal of Structural Engineering 123(7), 880–888 (1997)CrossRefGoogle Scholar
  34. 34.
    Adeli, H., Wu, M.: Regularization neural network for construction cost estimation. Journal of Construction Engineering and Management 124(1), 18–24 (1998)CrossRefGoogle Scholar
  35. 35.
    Adeli, H., Samant, A.: An adaptive conjugate gradient neural network - wavelet model for traffic incident detection. Computer-Aided Civil and Infrastructure Engineering 15(4), 251–260 (2000)CrossRefGoogle Scholar
  36. 36.
    Sirca, G., Adeli, H.: Neural network model for uplift load capacity of metal roof panels. Journal of Structural Engineering 127(11), 1276–1285 (2001)CrossRefGoogle Scholar
  37. 37.
    Sirca, G., Adeli, H.: Neural network model for uplift load capacity of metal roof panels - closure. Journal of Structural Engineering 129(4), 562–563 (2003)CrossRefGoogle Scholar
  38. 38.
    Dharia, A., Adeli, H.: Neural network model for rapid forecasting of freeway link travel time. Engineering Applications of Artificial Intelligence 16(7-8), 607–613 (2003)CrossRefGoogle Scholar
  39. 39.
    Panakkat, A., Adeli, H.: Neural network models for earthquake magnitude prediction using multiple seismicity indicators. International Journal of Neural Systems 17(1), 13–33 (2007)CrossRefGoogle Scholar
  40. 40.
    Jiang, X., Adeli, H.: Neuro-genetic algorithm for nonlinear active control of highrise buildings. International Journal for Numerical Methods in Engineering 75(8), 770–786 (2008)CrossRefGoogle Scholar
  41. 41.
    Kandel, E.R., Schwartz, J.H., Jessell, T.M.: Principles of Neural Science, 4th edn. McGraw-Hill, New York (2000)Google Scholar
  42. 42.
    Hodgkin, A.L., Huxley, A.F.: A quantitative description of ion currents and its applications to conduction and excitation in nerve membranes. Journal of Physiology 117, 500–544 (1952)Google Scholar
  43. 43.
    Rinzel, J., Ermentrout, G.B.: Analysis of neuronal excitability and oscillations. In: Koch, C., Segev, I. (eds.) Methods in Neuronal Modeling, pp. 135–169. MIT Press, Cambridge (1989)Google Scholar
  44. 44.
    Hille, B.: Ionic channels of excitable membranes, 2nd edn. Sinauer Associates, Sunderland (1992)Google Scholar
  45. 45.
    Ermentrout, G.B.: Type I membranes, phase resetting curves, and synchrony. Neural Computation 8, 979–1001 (1996)CrossRefGoogle Scholar
  46. 46.
    Hoppensteadt, F.C., Izhikevich, E.M.: Weakly Connected Neural Networks. Springer, New York (1997)Google Scholar
  47. 47.
    Izhikevich, E.M.: Solving the distal reward problem through linkage of STDP and dopamine signaling. Cerebral Cortex 17, 2443–2452 (2007)CrossRefGoogle Scholar
  48. 48.
    Abbott, L.F., Kepler, T.B.: Model neurons: From Hodgkin-Huxley to Hopfield. In: Garrido, L. (ed.) Statistical Mechanics of Neural Networks. Springer, Berlin (1990)Google Scholar
  49. 49.
    Kepler, T.B., Abbott, L.F., Marder, E.: Reduction of conductance-based neuron models. Biological Cybernetics 66, 381–387 (1992)zbMATHCrossRefGoogle Scholar
  50. 50.
    Ermentrout, G.B., Kopell, N.: Parabolic bursting in an excitable system coupled with a slow oscillation. SIAM Journal on Applied Mathematics 46, 233–253 (1986)zbMATHCrossRefMathSciNetGoogle Scholar
  51. 51.
    Gerstner, W.: Time structure of the activity in neural network models. Physical Review E 51, 738–758 (1995)CrossRefGoogle Scholar
  52. 52.
    Kistler, W.M., Gerstner, W., van Hemmen, J.L.: Reduction of Hodgkin-Huxley equations to a single-variable threshold model. Neural Computation 9, 1015–1045 (1997)CrossRefGoogle Scholar
  53. 53.
    Izhikevich, E.M.: Resonate-and-fire neurons. Neural Networks 14, 883–894 (2001)CrossRefGoogle Scholar
  54. 54.
    Gerstner, W., Kistler, W.M.: Spiking Neuron Models. Single Neurons, Populations, Plasticity. Cambridge University Press, New York (2002)zbMATHGoogle Scholar
  55. 55.
    Bohte, S.M., Kok, J.N., La Poutré, J.A.: Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing 48(1-4), 17–37 (2002)zbMATHCrossRefGoogle Scholar
  56. 56.
    Natschläger, T., Ruf, B.: Spatial and temporal pattern analysis via spiking neurons. Network: Computation in Neural Systems 9(3), 319–332 (1998)zbMATHCrossRefGoogle Scholar
  57. 57.
    Booij, O., Nguyen, H.T.: A gradient descent rule for multiple spiking neurons emitting multiple spikes. Information Processing Letters 95(6), 552–558 (2005)CrossRefMathSciNetGoogle Scholar
  58. 58.
    Ghosh-Dastidar, S., Adeli, H.: A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection. Neural Networks 22 (in press, 2009)Google Scholar
  59. 59.
    Xin, J., Embrechts, M.J.: Supervised learning with spiking neural networks. In: Proceedings of the International Joint Conference on Neural Networks, Washington, DC, vol. 3, pp. 1772–1777 (2001)Google Scholar
  60. 60.
    McKennoch, S., Liu, D., Bushnell, L.G.: Fast modifications of the SpikeProp algorithm. In: Proceedings of the International Joint Conference on Neural Networks, Vancouver, Canada, pp. 3970–3977 (2006)Google Scholar
  61. 61.
    Silva, S.M., Ruano, A.E.: Application of Levenberg-Marquardt method to the training of spiking neural networks. In: Proceedings of the International Conference on Neural Networks and Brain, vol. 3, pp. 1354–1358 (2005)Google Scholar
  62. 62.
    Fahlman, S.E.: Faster-learning variations of back-propagation: An empirical study. In: Proceedings of the 1988 Connectionist Models Summer School, San Mateo, CA, pp. 38–51. Morgan Kaufmann, San Francisco (1988)Google Scholar
  63. 63.
    Riedmiller, M., Braun, H.: A direct adaptive method for faster backpropagation learning: The Rprop algorithm. In: IEEE International Conference on Neural Networks, San Francisco, CA, vol. 1, pp. 586–591 (1993)Google Scholar
  64. 64.
    Schrauwen, B., van Campenhout, J.: Extending SpikeProp. In: Proceedings of the International Joint Conference on Neural Networks, Budapest, pp. 471–476 (2004)Google Scholar
  65. 65.
    Moore, S.C.: Backpropagation in spiking neural networks. Master’s thesis, University of Bath (2002)Google Scholar
  66. 66.
    Maass, W.: Fast sigmoidal networks via spiking neurons. Neural Computation 9(2), 279–304 (1997c)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Samanwoy Ghosh-Dastidar
    • 1
  • Hojjat Adeli
    • 2
  1. 1.Department of Biomedical EngineeringThe Ohio State University 
  2. 2.Abba G. Lichtenstein Professor, Departments of Biomedical Engineering, Biomedical Informatics, Civil and Environmental Engineering and Geodetic Science, Electrical and Computer Engineering, and NeuroscienceThe Ohio State UniversityColumbus

Personalised recommendations