Abstract
The so far developed and widely utilized connectionist systems (artificial neural networks) are mainly based on a single brain-like connectionist principle of information processing, where learning and information exchange occur in the connections. This paper extends this paradigm of connectionist systems to a new trend—integrative connectionist learning systems (ICOS) that integrate in their structure and learning algorithms principles from different hierarchical levels of information processing in the brain, including neuronal-, genetic-, quantum. Spiking neural networks (SNN) are used as a basic connectionist learning model which is further extended with other information learning principles to create different ICOS. For example, evolving SNN for multitask learning are presented and illustrated on a case study of person authentification based on multimodal auditory and visual information. Integrative gene-SNN are presented, where gene interactions are included in the functioning of a spiking neuron. They are applied on a case study of computational neurogenetic modeling. Integrative quantum-SNN are introduced with a quantum Hebbian learning, where input features as well as information spikes are represented by quantum bits that result in exponentially faster feature selection and model learning. ICOS can be used to solve more efficiently challenging biological and engineering problems when fast adaptive learning systems are needed to incrementally learn in a large dimensional space. They can also help to better understand complex information processes in the brain especially how information processes at different information levels interact. Open questions, challenges and directions for further research are presented.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Arbib M (ed) (2003) The handbook of brain theory and neural networks. MIT Press, Cambridge
Ashburner M, Ball CA et al (2000) Gene ontology: tool for the unification of biology. The gene ontology consortium. Nat Genet 25:25–29
Benuskova L, Kasabov N (2007a) Computational neurogenetic modeling. Springer, New York
Benuskova L, Kasabov N (2007b) Modeling L-LTP based on changes in concentration of pCREB transcription factor. Neurocomputing 70(10–12):2035–2040
Benuskova L, Jain V et al (2006) Computational neurogenetic modelling: a pathway to new discoveries in genetic neuroscience. Int J Neural Syst 16(3):215–227
Carpenter G, Grossberg S (1990) ART3: hierarchical search using chemical transmitters in self-organising pattern-recognition architectures. Neural Netw 3(2):129–152
Carpenter GA, Grossberg S (1992) Neural networks for vision and image processing. MIT Press, Cambridge
Carpenter G, Grossberg S et al (1991) Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analogue multi-dimensional maps. IEEE Trans Neural Netw 3(5):698–713
Chalmers DJ (1996) The conscious mind: in search of a fundamental theory. Oxford University Press, Oxford
Collado-Vides J, Hofestadt R (eds) (2002) Gene regulation and metabolism. Post-genomic computational approaches. MIT Press, Cambridge
Cooper LN, Intrator N et al (2004) Theory of cortical plasticity. World Scientific Publishing, Singapore
Crick F (1970) Central dogma of molecular biology. Nature 227:561–563
Darwin C (1859) The origin of species by means of natural selection. John Murray, London
Defoin-Platel M, Schliebs S et al (2007a) Quantum inspired feature selection and neural network models for time series prediction. IJCNN
Defoin-Platel M, Schliebs S et al (2007b) A versatile quantum inspired evolutionary algorithm. In: IEEE Congress on evolutionary computation. IEEE Press, Singapore
Dow J, Lindsay G et al (1995) Biochemistry molecules, cells and the body. Addison-Wesley, Boston
Ezhov A, Ventura D (2000) Quantum neural networks. In: Kasabov N (ed) Future directions for intelligent systems and information sciences. Springer Verlag
Feynman RP, Leighton RB et al (1965) The feynman lectures on physics. Addison-Wesley Publishing Company, Massachusetts
Fogel G, Corne D (2003) Evolutionary computation for bioinformatics. Morgan Kaufmann Publ
Fogel D, Fogel L et al (1990) Evolving neural networks. Biol Cybern 63:487–493
Freeman WJ (2000) Neurodynamics. An exploration in mesoscopic brain dynamics. Springer-Verlag, London
Gerstner W, Kistler WM (2002) Spiking neuron models. Cambridge University Press, Cambridge
Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley, Reading
Gottgtroy P, Kasabov N et al (2006) Evolving ontologies for intelligent decision support. In: Sanches E (ed) Fuzzy logic and the semantic web. Elsevier, pp 415–439
Grossberg S (1982) Studies of mind and brain. Reidel, Boston
Grover LK (1996) A fast quantum mechanical algorithm for database search. In: Proceedings of the twenty-eighth annual ACM symposium on theory of computing, ACM Press, New York
Han K-H, Kim J-H (2002) Quantum-inspired evolutionary algorithm for a class of combinatorial optimization. IEEE Trans Evol Comput 6:580–593
Hey T (1999) Quantum computing: an introduction. Comput Control Eng J 10(6):105–112
Holland JH (1975) Adaptation in natural and artificial systems. The University of Michigan Press, Ann Arbor
Izhikevich EM (2003) Simple model of spiking neurons. IEEE Trans Neural Netw 14(6):1569–1572
Izhikevich EM, Desai NS (2003) Relating STDP to BCM. Neural Comput 15:1511–1523
Johnston S, Prasad G et al (2005) Comparative investigation into classical and spiking neuron implementations on FPGAs. Springer LNCS 3696:269–274
Kasabov N (1996) Foundations of neural networks, fuzzy systems and knowledge engineering. MIT Press
Kasabov N (2001) Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning. IEEE Trans Syst Man Cybern B Cybern 31(6):902–918
Kasabov NK (2002) Evolving connectionist systems. Methods and applications in bioinformatics, brain study and intelligent machines. Springer-Verlag, London
Kasabov N (2006) Brain-, gene-, and quantum inspired computational intelligence: challenges and opportunities. In: Reusch B (ed) Computational intelligence: theory and practice. Springer, Heiderlberg, pp 521–544
Kasabov N (2007) Evolving connectionist systems: the knowledge engineering approach. Springer, London
Kasabov N, Benuskova L (2004) Computational neurogenetics. J Comput Theor Nanosci 1(1):47–61
Kasabov N, Song Q (2002) DENFIS: dynamic, evolving neural-fuzzy inference systems and its application for time-series prediction. IEEE Trans Fuzzy Syst 10(2):144–154
Kasabov N, Jain V et al (2007) Evolving brain-gene ontology system (EBGOS): towards integrating bioinformatics and neuroinformatics data to facilitate discoveries. In: International joint conference on neural networks, IEEE Press, Orlando
Kohonen T (1997) Self-organizing maps. Springer Verlag
Narayanan A, Meneer T (2000) Quantum artificial neural network architectures and components. Inf Sci Appl 128:3–4
Penrose R (1994) Shadows of the mind: a search for the missing science of consciousness. Oxford University Press, Oxford
Sanderson C, Paliwal KK (2004) Identity verification using speech and face information. Digit Signal Process 14:449–480
Shor PW (1997) Polynomial-time algorithms for prime factorization and discrete logarithms on a quantum computer. SIAM J Comput 26:1484–1509
Taylor JG (1999) The race for consciousness. MIT Press, Cambridge
Venayagamoorthy GK, Gaurav S (2005) Quantum-inspired evolutionary algorithms and binary particle swarm optimization for training MLP and SRN neural networks. J Theor Comput Nanosci 2(4):561–568
Wysoski S, Benuskova L et al (2006) On-line learning with structural adaptation in a network of spiking neurons for visual pattern recognition. In: ICANN 2006. Springer-LNCS, Athens, Greece
Wysoski S, Benuskova L et al (2007a) Brain-like evolving spiking neural networks for multimodal information processing. In: ICONIP 2007, Kitakyushu, Japan, LNCS, Springer
Wysoski S, Benuskova L et al (2007b) Text-independent speaker authentication with spiking neural networks. In: Proc ICANN ’07, vol 4668, pp 758–767
Wysoski S, Benuskova L et al (2008) Fast and adaptive network of spiking for multi-view visual pattern recognition. Neurocomputing (in print)
Yao X (1993) Evolutionary artificial neural networks. Int J Neural Syst 4(3):203–222
Acknowledgements
The paper is supported partially by a grant AUTX0201 funded by the Foundation of Research Science and Technology of New Zealand and also by the Auckland University of Technology grant to the Knowledge Engineering and Discovery Research Institute KEDRI (http://www.kedri.info). I acknowledge the contribution to different experiments by Simei Wysoski, Stefan Shliebs, Michael Defoin-Platel, Lubica Benuskova from KEDRI and the inspirational discussions I had with Dr Liam McDaid from the University of Ulster during my visits.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kasabov, N. Integrative connectionist learning systems inspired by nature: current models, future trends and challenges. Nat Comput 8, 199–218 (2009). https://doi.org/10.1007/s11047-008-9066-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11047-008-9066-z