Natural Computing

, Volume 8, Issue 2, pp 199–218 | Cite as

Integrative connectionist learning systems inspired by nature: current models, future trends and challenges

Article

Abstract

The so far developed and widely utilized connectionist systems (artificial neural networks) are mainly based on a single brain-like connectionist principle of information processing, where learning and information exchange occur in the connections. This paper extends this paradigm of connectionist systems to a new trend—integrative connectionist learning systems (ICOS) that integrate in their structure and learning algorithms principles from different hierarchical levels of information processing in the brain, including neuronal-, genetic-, quantum. Spiking neural networks (SNN) are used as a basic connectionist learning model which is further extended with other information learning principles to create different ICOS. For example, evolving SNN for multitask learning are presented and illustrated on a case study of person authentification based on multimodal auditory and visual information. Integrative gene-SNN are presented, where gene interactions are included in the functioning of a spiking neuron. They are applied on a case study of computational neurogenetic modeling. Integrative quantum-SNN are introduced with a quantum Hebbian learning, where input features as well as information spikes are represented by quantum bits that result in exponentially faster feature selection and model learning. ICOS can be used to solve more efficiently challenging biological and engineering problems when fast adaptive learning systems are needed to incrementally learn in a large dimensional space. They can also help to better understand complex information processes in the brain especially how information processes at different information levels interact. Open questions, challenges and directions for further research are presented.

Keywords

Connectionist learning systems Artificial neural networks Spiking neural networks Evolving spiking neural networks Multiple task learning Multimodal audio-visual information processing Gene-spiking neural networks Computational neurogenetic modeling Quantum spiking neural networks 

References

  1. Arbib M (ed) (2003) The handbook of brain theory and neural networks. MIT Press, CambridgeMATHGoogle Scholar
  2. Ashburner M, Ball CA et al (2000) Gene ontology: tool for the unification of biology. The gene ontology consortium. Nat Genet 25:25–29CrossRefGoogle Scholar
  3. Benuskova L, Kasabov N (2007a) Computational neurogenetic modeling. Springer, New YorkGoogle Scholar
  4. Benuskova L, Kasabov N (2007b) Modeling L-LTP based on changes in concentration of pCREB transcription factor. Neurocomputing 70(10–12):2035–2040CrossRefGoogle Scholar
  5. Benuskova L, Jain V et al (2006) Computational neurogenetic modelling: a pathway to new discoveries in genetic neuroscience. Int J Neural Syst 16(3):215–227CrossRefGoogle Scholar
  6. Carpenter G, Grossberg S (1990) ART3: hierarchical search using chemical transmitters in self-organising pattern-recognition architectures. Neural Netw 3(2):129–152CrossRefGoogle Scholar
  7. Carpenter GA, Grossberg S (1992) Neural networks for vision and image processing. MIT Press, CambridgeGoogle Scholar
  8. Carpenter G, Grossberg S et al (1991) Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analogue multi-dimensional maps. IEEE Trans Neural Netw 3(5):698–713CrossRefGoogle Scholar
  9. Chalmers DJ (1996) The conscious mind: in search of a fundamental theory. Oxford University Press, OxfordMATHGoogle Scholar
  10. Collado-Vides J, Hofestadt R (eds) (2002) Gene regulation and metabolism. Post-genomic computational approaches. MIT Press, CambridgeGoogle Scholar
  11. Cooper LN, Intrator N et al (2004) Theory of cortical plasticity. World Scientific Publishing, SingaporeMATHGoogle Scholar
  12. Crick F (1970) Central dogma of molecular biology. Nature 227:561–563CrossRefGoogle Scholar
  13. Darwin C (1859) The origin of species by means of natural selection. John Murray, LondonGoogle Scholar
  14. Defoin-Platel M, Schliebs S et al (2007a) Quantum inspired feature selection and neural network models for time series prediction. IJCNNGoogle Scholar
  15. Defoin-Platel M, Schliebs S et al (2007b) A versatile quantum inspired evolutionary algorithm. In: IEEE Congress on evolutionary computation. IEEE Press, SingaporeGoogle Scholar
  16. Dow J, Lindsay G et al (1995) Biochemistry molecules, cells and the body. Addison-Wesley, BostonGoogle Scholar
  17. Ezhov A, Ventura D (2000) Quantum neural networks. In: Kasabov N (ed) Future directions for intelligent systems and information sciences. Springer VerlagGoogle Scholar
  18. Feynman RP, Leighton RB et al (1965) The feynman lectures on physics. Addison-Wesley Publishing Company, MassachusettsMATHGoogle Scholar
  19. Fogel G, Corne D (2003) Evolutionary computation for bioinformatics. Morgan Kaufmann PublGoogle Scholar
  20. Fogel D, Fogel L et al (1990) Evolving neural networks. Biol Cybern 63:487–493CrossRefGoogle Scholar
  21. Freeman WJ (2000) Neurodynamics. An exploration in mesoscopic brain dynamics. Springer-Verlag, LondonMATHGoogle Scholar
  22. Gerstner W, Kistler WM (2002) Spiking neuron models. Cambridge University Press, CambridgeMATHGoogle Scholar
  23. Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley, ReadingMATHGoogle Scholar
  24. Gottgtroy P, Kasabov N et al (2006) Evolving ontologies for intelligent decision support. In: Sanches E (ed) Fuzzy logic and the semantic web. Elsevier, pp 415–439Google Scholar
  25. Grossberg S (1982) Studies of mind and brain. Reidel, BostonMATHGoogle Scholar
  26. Grover LK (1996) A fast quantum mechanical algorithm for database search. In: Proceedings of the twenty-eighth annual ACM symposium on theory of computing, ACM Press, New YorkGoogle Scholar
  27. Han K-H, Kim J-H (2002) Quantum-inspired evolutionary algorithm for a class of combinatorial optimization. IEEE Trans Evol Comput 6:580–593CrossRefGoogle Scholar
  28. Hey T (1999) Quantum computing: an introduction. Comput Control Eng J 10(6):105–112CrossRefGoogle Scholar
  29. Holland JH (1975) Adaptation in natural and artificial systems. The University of Michigan Press, Ann ArborGoogle Scholar
  30. Izhikevich EM (2003) Simple model of spiking neurons. IEEE Trans Neural Netw 14(6):1569–1572CrossRefMathSciNetGoogle Scholar
  31. Izhikevich EM, Desai NS (2003) Relating STDP to BCM. Neural Comput 15:1511–1523MATHCrossRefGoogle Scholar
  32. Johnston S, Prasad G et al (2005) Comparative investigation into classical and spiking neuron implementations on FPGAs. Springer LNCS 3696:269–274Google Scholar
  33. Kasabov N (1996) Foundations of neural networks, fuzzy systems and knowledge engineering. MIT PressGoogle Scholar
  34. Kasabov N (2001) Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning. IEEE Trans Syst Man Cybern B Cybern 31(6):902–918CrossRefGoogle Scholar
  35. Kasabov NK (2002) Evolving connectionist systems. Methods and applications in bioinformatics, brain study and intelligent machines. Springer-Verlag, LondonMATHGoogle Scholar
  36. Kasabov N (2006) Brain-, gene-, and quantum inspired computational intelligence: challenges and opportunities. In: Reusch B (ed) Computational intelligence: theory and practice. Springer, Heiderlberg, pp 521–544CrossRefGoogle Scholar
  37. Kasabov N (2007) Evolving connectionist systems: the knowledge engineering approach. Springer, LondonMATHGoogle Scholar
  38. Kasabov N, Benuskova L (2004) Computational neurogenetics. J Comput Theor Nanosci 1(1):47–61CrossRefGoogle Scholar
  39. Kasabov N, Song Q (2002) DENFIS: dynamic, evolving neural-fuzzy inference systems and its application for time-series prediction. IEEE Trans Fuzzy Syst 10(2):144–154CrossRefGoogle Scholar
  40. Kasabov N, Jain V et al (2007) Evolving brain-gene ontology system (EBGOS): towards integrating bioinformatics and neuroinformatics data to facilitate discoveries. In: International joint conference on neural networks, IEEE Press, OrlandoGoogle Scholar
  41. Kohonen T (1997) Self-organizing maps. Springer VerlagGoogle Scholar
  42. Narayanan A, Meneer T (2000) Quantum artificial neural network architectures and components. Inf Sci Appl 128:3–4Google Scholar
  43. Penrose R (1994) Shadows of the mind: a search for the missing science of consciousness. Oxford University Press, OxfordGoogle Scholar
  44. Sanderson C, Paliwal KK (2004) Identity verification using speech and face information. Digit Signal Process 14:449–480CrossRefGoogle Scholar
  45. Shor PW (1997) Polynomial-time algorithms for prime factorization and discrete logarithms on a quantum computer. SIAM J Comput 26:1484–1509MATHCrossRefMathSciNetGoogle Scholar
  46. Taylor JG (1999) The race for consciousness. MIT Press, CambridgeGoogle Scholar
  47. Venayagamoorthy GK, Gaurav S (2005) Quantum-inspired evolutionary algorithms and binary particle swarm optimization for training MLP and SRN neural networks. J Theor Comput Nanosci 2(4):561–568CrossRefGoogle Scholar
  48. Wysoski S, Benuskova L et al (2006) On-line learning with structural adaptation in a network of spiking neurons for visual pattern recognition. In: ICANN 2006. Springer-LNCS, Athens, GreeceGoogle Scholar
  49. Wysoski S, Benuskova L et al (2007a) Brain-like evolving spiking neural networks for multimodal information processing. In: ICONIP 2007, Kitakyushu, Japan, LNCS, SpringerGoogle Scholar
  50. Wysoski S, Benuskova L et al (2007b) Text-independent speaker authentication with spiking neural networks. In: Proc ICANN ’07, vol 4668, pp 758–767Google Scholar
  51. Wysoski S, Benuskova L et al (2008) Fast and adaptive network of spiking for multi-view visual pattern recognition. Neurocomputing (in print)Google Scholar
  52. Yao X (1993) Evolutionary artificial neural networks. Int J Neural Syst 4(3):203–222CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2008

Authors and Affiliations

  1. 1.Knowledge Engineering and Discovery Research Institute, KEDRIAuckland University of TechnologyAucklandNew Zealand

Personalised recommendations