Cognitive Computation

, Volume 1, Issue 1, pp 77–90 | Cite as

Cognitive Computation with Autonomously Active Neural Networks: An Emerging Field

Article

Abstract

The human brain is autonomously active. To understand the functional role of this self-sustained neural activity, and its interplay with the sensory data input stream, is an important question in cognitive system research and we review here the present state of theoretical modeling. This review will start with a brief overview of the experimental efforts, together with a discussion of transient versus self-sustained neural activity in the framework of reservoir computing. The main emphasis will be then on two paradigmal neural network architectures showing continuously ongoing transient-state dynamics: saddle point networks and networks of attractor relics. Self-active neural networks are confronted with two seemingly contrasting demands: a stable internal dynamical state and sensitivity to incoming stimuli. We show, that this dilemma can be solved by networks of attractor relics based on competitive neural dynamics, where the attractor relics compete on one side with each other for transient dominance, and on the other side with the dynamical influence of the input signals. Unsupervised and local Hebbian-style online learning then allows the system to build up correlations between the internal dynamical transient states and the sensory input stream. An emergent cognitive capability results from this set-up. The system performs online, and on its own, a nonlinear independent component analysis of the sensory data stream, all the time being continuously and autonomously active. This process maps the independent components of the sensory input onto the attractor relics, which acquire in this way a semantic meaning.

Keywords

Recurrent neural networks Autonomous neural dynamics Transient state dynamics Emergent cognitive capabilities 

References

  1. 1.
    Gros C. Complex and adaptive dynamical systems, a primer. Berlin: Springer; 2008.Google Scholar
  2. 2.
    Gros C. Emotions, diffusive emotional control and the motivational problem for autonomous cognitive systems. In: Vallverdu J, Casacuberta D (eds) Handbook of research on synthetic emotions and sociable robotics: new applications in affective computing and artificial intelligence. IGI-Global; 2009 (in press).Google Scholar
  3. 3.
    Fox MD, Raichle ME. Spontaneous fluctuations in brain activity observed with functional magnetic resonance imaging. Nat Rev Neurosci. 2007;9:700–11.CrossRefGoogle Scholar
  4. 4.
    Arieli A, Sterkin A, Grinvald A, Aertsen A. Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses. Science. 1996;273:1868–71.PubMedCrossRefGoogle Scholar
  5. 5.
    Raichle ME, Mintun MA. Brain work and brain imaging. Annl Rev Neurosci. 2006;29:449–76.CrossRefGoogle Scholar
  6. 6.
    Vogels TP, Rajan K, Abbott LF. Neural network dynamics. Annl Rev Neurosci. 2005;28:357–76.CrossRefGoogle Scholar
  7. 7.
    Fiser J, Chiu C, Weliky M. Small modulation of ongoing cortical dynamics by sensory input during natural vision. Nature. 2004;431:573–8.PubMedCrossRefGoogle Scholar
  8. 8.
    Fox MD, Corbetta M, Snyder AZ, Vincent JL, Raichle ME. Spontaneous neuronal activity distinguishes human dorsal and ventral attention systems. Proc Natl Acad Sci. 2003;103:10046–51.CrossRefGoogle Scholar
  9. 9.
    Kenet T, Bibitchkov D, Tsodyks M, Grinvald A, Arieli A. Spontaneously emerging cortical representations of visual attributes. Nature. 2003;425:954–6.PubMedCrossRefGoogle Scholar
  10. 10.
    Ringach DL. States of mind. Nature. 2003;425:912–3.PubMedCrossRefGoogle Scholar
  11. 11.
    MacLean JN, Watson BO, Aaron GB, Yuste R. Internal dynamics determine the cortical response to thalamic stimulation. Neuron. 2005;48:811–23.PubMedCrossRefGoogle Scholar
  12. 12.
    Fox MD, Snyder AZ, Vincent JL, Corbetta M, Van Essen DC, Raichle ME. The human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proc Natl Acad Sci. 2005;102:9673–8.PubMedCrossRefGoogle Scholar
  13. 13.
    Abeles M, Bergman H, Gat I, Meilijson I, Seidemann E, Tishby N, et al. Cortical activity flips among quasi-stationary states. Proc Natl Acad Sci. 1995;92:8616–20.PubMedCrossRefGoogle Scholar
  14. 14.
    Gros C. Self-sustained thought processes in a dense associative network. In: Furbach, U (ed) KI 2005: Advances in artificial intelligence, Springer lecture notes in artificial intelligence 3698; 2005. p. 366–79.Google Scholar
  15. 15.
    Edelman GM, Tononi GA. A universe of consciousness. New York: Basic Books; 2000.Google Scholar
  16. 16.
    Edelman GM. Naturalizing consciousness: a theoretical framework. Proc Natl Acad Sci. 2003;100:5520–4.PubMedCrossRefGoogle Scholar
  17. 17.
    Baars BJ, Ramsoy TZ, Laureys S. Brain, conscious experience and the observing self. Trend Neurosci. 2003;26:671–5.PubMedCrossRefGoogle Scholar
  18. 18.
    Morcom AM, Fletcher PC. Does the brain have a baseline? Why we should be resisting a rest. Neuroimage. 2007;37:1073–82.CrossRefGoogle Scholar
  19. 19.
    Vincent JL, et al. Intrinsic functional architecture in the anaesthetized monkey brain. Nature. 2007;447:83–6.PubMedCrossRefGoogle Scholar
  20. 20.
    Greicius MD, et al. Persistent default-mode network connectivity during light sedation. Human Brain Map. 2008;29:839–47.CrossRefGoogle Scholar
  21. 21.
    Pagnoni G, Cekic M, Guo Y. Thinking about not-thinking: neural correlates of conceptual processing during Zen meditation. PLoS. 2008;3:1–10.Google Scholar
  22. 22.
    Jaeger H. The “echo State”: approach to analysing and training recurrent neural networks. GMD-Forschungszentrum Informationstechnik; 2001.Google Scholar
  23. 23.
    Jaeger H, Haas H. Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science. 2004;304:78–80.PubMedCrossRefGoogle Scholar
  24. 24.
    Maass W, Natschlager T, Markram H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 2002;14:2531–60.PubMedCrossRefGoogle Scholar
  25. 25.
    Maass W, Markram H. On the computational power of recurrent circuits of spiking neurons. J Comput Syst Sci. 2004;69:593–616.CrossRefGoogle Scholar
  26. 26.
    Maass W, Joshi P, Sontag ED. Computational aspects of feedback in neural circuits. PLoS Comput Biol. 2007;3:e165.PubMedCrossRefGoogle Scholar
  27. 27.
    Baars BJ, Franklin S. How conscious experience and working memory interact. Trend Coginit Sci. 2003;7:166–72.CrossRefGoogle Scholar
  28. 28.
    Dehaene S, Naccache L. Towards a cognitive neuroscience of consciousness: basic evidence and a workspace framework. Cognition. 2003;79:1–37.CrossRefGoogle Scholar
  29. 29.
    Shanahan M. A spiking neuron model of cortical broadcast and competition. Conscious Cognit. 2003;17:288–303.CrossRefGoogle Scholar
  30. 30.
    Shadlen MN, Newsome WT. Noise, neural codes and cortical organization. Find Curr Opin Cognit Neurosci. 1998;4:569–79.Google Scholar
  31. 31.
    Eggermont JJ. Is there a neural code? Neurosci Biobehav Rev. 1998;22:355–70.PubMedCrossRefGoogle Scholar
  32. 32.
    Averbeck BB, Lee D. Coding and transmission of information by neural ensembles. Trend Neurosci. 2004;27:225–30.PubMedCrossRefGoogle Scholar
  33. 33.
    VanRullen R, Koch C. Is perception discrete or continuous? Trend Cognit Sci. 2003;5:207–13.CrossRefGoogle Scholar
  34. 34.
    Kline K, Holcombe AO, Eagleman DM. Illusory motion reversal is caused by rivalry, not by perceptual snapshots of the visual field. Vision Res. 2004;44:2653–8.PubMedCrossRefGoogle Scholar
  35. 35.
    VanRullen R. The continuous wagon wheel illusion is object-based. Vision Res. 2006;46:4091–5.PubMedCrossRefGoogle Scholar
  36. 36.
    Rabinovich M, Huerta R, Varona P, Afraimovich VS. Transient cognitive dynamics, metastability, and decision making. PLoS Comput Biol. 2008;4:e1000072.PubMedCrossRefGoogle Scholar
  37. 37.
    Krupa M. Robust heteroclinic cycles. J Nonlinear Sci. 1997;7:129–76.CrossRefGoogle Scholar
  38. 38.
    Rabinovich M, Volkovskii A, Lecanda P, Huerta R, Abarbanel HDI, Laurent G. Dynamical encoding by networks of competing neuron groups: winnerless competition. Phys Rev Lett. 2001;87:068102.PubMedCrossRefGoogle Scholar
  39. 39.
    Amit DJ. Modeling brain function: the world of attractor neural networks. New York, NY, USA: Cambridge University Press; 1989.Google Scholar
  40. 40.
    Hasselmo ME, McClelland JL. Neural models of memory. Curr Opin Neurobiol. 1999;9:184–8.PubMedCrossRefGoogle Scholar
  41. 41.
    Tsuda I. Toward an interpretation of dynamic neural activity in terms of chaotic dynamical systems. Behav Brain Sci. 2002;24:793–810.CrossRefGoogle Scholar
  42. 42.
    Horn D, Usher M. Neural networks with dynamical thresholds. Phys Rev A. 1989;40:1036–44.PubMedCrossRefGoogle Scholar
  43. 43.
    Sompolinsky H, Kanter I. Temporal association in asymmetric neural networks. Phys Rev Lett. 1986;57:2861–4.PubMedCrossRefGoogle Scholar
  44. 44.
    Gros C. Neural networks with transient state dynamics. New J Phys. 2007;9:109.CrossRefGoogle Scholar
  45. 45.
    Maass W. On the computational power of winner-take-all. Neural Comput. 2000;12:2519–35.PubMedCrossRefGoogle Scholar
  46. 46.
    O’Reilly RC. Six principles for biologically based computational models of cortical cognition. Trend Cognit Sci. 1998;2:455–62.CrossRefGoogle Scholar
  47. 47.
    Crick FC, Koch C. A framework for consciousness. Nat Neurosci. 2003;6:119–26.PubMedCrossRefGoogle Scholar
  48. 48.
    Koch C. The quest for consciousness—a neurobiological approach. Robert and Company; 2004.Google Scholar
  49. 49.
    Quiroga RQ, Kreiman G, Koch C, Fried I. Sparse but not grandmother-cell coding in the medial temporal lobe. Trend Cognit Sci. 2008;12:87–91.CrossRefGoogle Scholar
  50. 50.
    Olshausen BA, Field DJ. Sparse coding of sensory inputs. Curr Opin Neurobiol. 2004;14:481–7.PubMedCrossRefGoogle Scholar
  51. 51.
    Lin L, Osan R, Shoham S, Jin W, Zuo W, Tsien JZ. Identification of network-level coding units for real-time representation of episodic experiences in the hippocampus. Proc Natl Acad Sci. 2005;102:6125–613.PubMedCrossRefGoogle Scholar
  52. 52.
    Lin L, Osan R, Tsien JZ. Organizing principles of real-time memory encoding: neural clique assemblies and universal neural codes. Trend Neurosci. 2006;29:48–57.PubMedCrossRefGoogle Scholar
  53. 53.
    Quiroga RQ, Reddy L, Kreiman G, Koch C, Fried I. Invariant visual representation by single neurons in the human brain. Nature. 2005;435:1102–7.PubMedCrossRefGoogle Scholar
  54. 54.
    Nelson DL, McEvoy CL, Schreiber TA. The University of South Florida free association, rhyme, and word fragment norms. Behav Res Method Instr Comput. 2004;36:402–7.Google Scholar
  55. 55.
    Palla G, Derényi I, Farkas I, Vicsek T. Uncovering the overlapping community structure of complex networks in nature and society. Nature. 2005;435:814–8.PubMedCrossRefGoogle Scholar
  56. 56.
    Riesenhuber M, Poggio T. Hierarchical models of object recognition in cortex. Nat Neurosci. 1999;2:1019–25.PubMedCrossRefGoogle Scholar
  57. 57.
    Kaczor G, Gros C. Evolving complex networks with conserved clique distributions. Phys Rev E. 2008;78:016107.Google Scholar
  58. 58.
    Gros C, Kaczor G. Semantic learning in autonomously active recurrent neural networks. Preprint.Google Scholar
  59. 59.
    Arbib MA. The handbook of brain theory and neural networks. Cambridge, MA: MIT Press; 2002.Google Scholar
  60. 60.
    Von der Malsburg C. The what and why of binding: the modeler’s perspective. Neuron. 1999;24:95–104.PubMedCrossRefGoogle Scholar
  61. 61.
    Singer W, Gray CM. Visual feature integration and the temporal correlation hypothesis. Annl Rev Neurosci. 1995;18:555–86.CrossRefGoogle Scholar
  62. 62.
    Berns GS, Cohen JD, Mintun MA. Brain regions responsive to novelty in the absence of awareness. Science. 1997;276:1272–5.PubMedCrossRefGoogle Scholar
  63. 63.
    Barceló F, Periáñez JA, Knight RT. Think differently: a brain orienting response to task novelty. NeuroReport. 2002;13:1887–92.PubMedCrossRefGoogle Scholar
  64. 64.
    Wise RA. Dopamine, learning and motivation. Nat Rev Neurosci. 2004;4:483–94.CrossRefGoogle Scholar
  65. 65.
    Redgrave P, Gurney K. The short-latency dopamine signal: a role in discovering novel actions? Nat Rev Neurosci. 2006;7:967–75.PubMedCrossRefGoogle Scholar
  66. 66.
    Bienenstock EL, Cooper LN, Munro PW. Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex. J Neurosci. 1982;2:32–48.PubMedGoogle Scholar
  67. 67.
    Gros C, Kaczor G. Learning in cognitive systems with autonomous dynamics. In: Proceedings of the 2008 international conference on cognitive systems, Karlsruhe; 2008.Google Scholar
  68. 68.
    Hyvärinen A, Oja E. Independent component analysis: algorithms and applications. Neural Netw. 2000;13:411–30.PubMedCrossRefGoogle Scholar
  69. 69.
    Choi S, Cichocki A, Park HM, Lee SY. Blind source separation and independent component analysis: a review. Neural Inform Process. 2005;6:1–57.Google Scholar
  70. 70.
    Földiák P. Forming sparse representations by local anti-Hebbian learning. Biol Cybernet. 1990;64:165–70.CrossRefGoogle Scholar
  71. 71.
    Butko N, Triesch J. Learning sensory representations with intrinsic plasticity. Neurocomputing. 2007;70:1130–8.CrossRefGoogle Scholar
  72. 72.
    Chialvo DR, Bak P. Learning from mistakes. Neuroscience. 1999;90:1137–48.PubMedCrossRefGoogle Scholar
  73. 73.
    Haykin S. Neural networks: a comprehensive foundation. Upper Saddle River, NJ: Prentice Hall; 1994.Google Scholar
  74. 74.
    Dreyfus G. Neural networks: methodology and applications. Berlin: Springer; 2005.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Institute of Theoretical PhysicsJ.W. Goethe University FrankfurtFrankfurt am MainGermany

Personalised recommendations