Skip to main content
Log in

Supervised and Unsupervised Learning with Two Sites of Synaptic Integration

Journal of Computational Neuroscience Aims and scope Submit manuscript

Abstract

Many learning rules for neural networks derive from abstract objective functions. The weights in those networks are typically optimized utilizing gradient ascent on the objective function. In those networks each neuron needs to store two variables. One variable, called activity, contains the bottom-up sensory-fugal information involved in the core signal processing. The other variable typically describes the derivative of the objective function with respect to the cell's activity and is exclusively used for learning. This variable allows the objective function's derivative to be calculated with respect to each weight and thus the weight update. Although this approach is widely used, the mapping of such two variables onto physiology is unclear, and these learning algorithms are often considered biologically unrealistic. However, recent research on the properties of cortical pyramidal neurons shows that these cells have at least two sites of synaptic integration, the basal and the apical dendrite, and are thus appropriately described by at least two variables. Here we discuss whether these results could constitute a physiological basis for the described abstract learning rules. As examples we demonstrate an implementation of the backpropagation of error algorithm and a specific self-supervised learning algorithm using these principles. Thus, compared to standard, one-integration-site neurons, it is possible to incorporate interesting properties in neural networks that are inspired by physiology with a modest increase of complexity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Agmon-Snir H, Carr CE, Rinzel J (1998) The role of dendrites in auditory coincidence detection. Nature 393: 268-272.

    Google Scholar 

  • Amitai Y, Friedman A, Connors BW, Gutnick MJ (1993) Regenerative activity in apical dendrites of pyramidal cells in neocortex. Cereb. Cortex 3: 26-38.

    Google Scholar 

  • Arbib MA (1998) The Handbook of Brain Theory and Neural Networks. MIT Press: Cambridge, MA.

    Google Scholar 

  • Artola A, Bröcher S, Singer W (1990) Different voltage-dependent thresholds for inducing long-term depression and long-term potentiation in slices of rat visual cortex. Nature 347: 69-72.

    Google Scholar 

  • Bartlett MS, Sejnowski TJ (1998) Viewpoint invariant face representation from visual experience by temporal association. In: Wechsler et al., eds. Face Recognition: From Theory to Applications. Springer, Berlin.

    Google Scholar 

  • Becker S (1996) Models of cortical self-organisation. Network: Comput. Neural Syst. 7: 7-31.

    Google Scholar 

  • Becker S, Hinton GE (1992) Self-organizing neural network that discovers surfaces in random-dot stereograms. Nature 355: 161-163.

    Google Scholar 

  • Berger T, Larkum ME, Lüscher H-R (2001) A high Ih channel density in the distal apical dendrite of layer 5 neocortical pyramidal cells increases bidirectional attenuation of EPSPs. J. Neurophysiol. 85: 855-868.

    Google Scholar 

  • Buzsaki G, Kandel A (1998) Somadendritic backpropagation of action potentials in cortical pyramidal cells of the awake rat. J. Neurophysiol. 79: 1587-1591.

    Google Scholar 

  • Carnevale NT, Tsai KY, Claiborne BJ, Brown TH (1997) Comparative electrotonic analysis of three classes of rat hippocampal neurons. J. Neurophysiol. 78: 703-720.

    Google Scholar 

  • Cash S, Yuste R (1998) Input summation by cultured pyramidal neurons is linear and position-independent. J. Neurosci. 18: 10-15.

    Google Scholar 

  • Cauller LJ, Connors BW (1994) Synaptic physiology of horizontal afferents to layer I in slices of rat SI neocortex. J. Neurosci. 14: 751-762.

    Google Scholar 

  • Cook EP, Johnston D (1999) Voltage-dependent properties of dendrites that eliminate location-dependent variability of synaptic input. J. Neurophysiol. 81: 535-543.

    Google Scholar 

  • Crick FHC (1989) The recent excitement about neural networks. Nature 337: 129-132.

    Google Scholar 

  • De Sa VR, Ballard DH (1998) Category learning through multimodality sensing. Neur. Comput. 10: 1097-1117.

    Google Scholar 

  • Dudek SM, Bear MF (1993) Bidirectional long-term modification of synaptic effectiveness in the adult and immature hippocampus. J. Neurosci. 13: 2910-2918.

    Google Scholar 

  • Földiak P (1991) Learning invariance from transformation sequences. Neur. Comput. 3: 194-200.

    Google Scholar 

  • Frey, B. (1998) Graphical Models for Machine Learning and Digital Communication. MIT Press: Cambridge, MA.

    Google Scholar 

  • Gupta A, Wang Y, Markram H (2000) Organizing principles for a diversity of GABAergic interneurons and synapses in the neocortex. Science 287: 273-278.

    Google Scholar 

  • Helmchen F, Svoboda K, Denk W, Tank DW (1999) In vivo dendritic calcium dynamics in deep-layer cortical pyramidal neurons. Nat. Neurosci. 2: 989-996.

    Google Scholar 

  • Hinton GE, McClelland JL (1988) Learning representations by recirculation. In: Anderson DZ, ed. Neural Information Processing Systems. American Institute of Physics, New York. pp. 358-366.

    Google Scholar 

  • Hirsch JA, Alonso JM, Reid RC (1995) Visually evoked calcium action potentials in cat striate cortex. Nature 378: 612-616.

    Google Scholar 

  • Johnston D, Magee JC, Colbert CM, Cristie BR (1996) Active properties of neuronal dendrites. Ann. Rev. Neurosci. 19: 165-186.

    Google Scholar 

  • Kay J, Floreano D, Phillips WA (1998) Contextually guided unsupervised learning using local multivariate binary processors. Neural Networks 11: 117-140.

    Google Scholar 

  • König P, Engel AK (1995) Correlated firing in sensory-motor systems. Curr. Opin. Neurobiol. 5: 511-519.

    Google Scholar 

  • Körding KP, König P (2000) Learning with two sites of synaptic integration. Network: Comput. Neural Syst. 11: 1-15.

    Google Scholar 

  • Larkum ME, Kaiser KM, Sakmann B (1999a) Calcium electrogenesis in distal apical dendrites of layer 5 pyramidal cells at a critical frequency of backpropagating action potentials. Proc. Natl. Acad. Sci. USA 96: 14600-14604.

    Google Scholar 

  • Larkum ME, Zhu JJ, Sakmann B (1999b) A new cellular mechanism for coupling inputs arriving at different cortical layers. Nature 398: 338-341.

    Google Scholar 

  • LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1990) Handwritten digit recognition with a backpropagation network. In: Touretzky D, ed. Advances in Neural Information Processing Systems 2. Morgan Kaufman: San Francisco.

  • Mainen ZF, Carnevale NT, Zador AM, Claiborne BJ, Brown TH (1996) Electrotonic architecture of hippocampal CA1 pyramidal neurons based on three-dimensional reconstructions. J. Neurophysiol. 76: 1904-1923.

    Google Scholar 

  • Mainen ZF, Sejnowski TJ (1996) Influence of dendritic structure on firing pattern in model neocortical neurons. Nature 382: 363-366.

    Google Scholar 

  • Markram H, Lübke J, Frotscher M, Sakmann B (1997) Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275: 213-215.

    Google Scholar 

  • Markram H, Wang Y, Tsodyks M (1998) Differential signaling via the same axon of neocortical pyramidal neurons. Proc. Natl. Acad. Sci. USA 98: 5323-5328.

    Google Scholar 

  • Mazzoni P, Andersen RA, Jordan MI (1991) A more biologically plausible learning rule for neural networks. Proc. Natl. Acad. Sci. USA 88: 4433-4437.

    Google Scholar 

  • Mel BW (1993) Synaptic integration in an excitable dendritic tree. J. Neurophysiol. 70: 1086-1101.

    Google Scholar 

  • Mel BW, Ruderman DL, Archie KA (1998) Translation-invariant orientation tuning in visual “complex” cells could derive from intradendritic computations. J. Neurosci. 18: 4325-4334.

    Google Scholar 

  • Minsky M, Papert S (1969) Perceptrons: An Introduction to Computational Geometry. MIT Press: Cambridge, MA.

  • O'Reilly RC (1996) Biologically plausible error-driven learning using local activation differences: The generalized recirculation algorithm. Neur. Comput. 8: 895-938.

    Google Scholar 

  • Pike FG, Meredith RM, Olding AWA, Paulsen O (1999) Postsynaptic bursting is essential for “Hebbian” induction of associative long-term potentiation at excitatory synapses in rat hippocampus. J. Phys. 518: 571-576.

    Google Scholar 

  • Rumelhart D, McCleland J (1986) Parallel Distributed Processing. Bradford Books, Cambridge, MA.

    Google Scholar 

  • Salin PA, Bullier J (1995) Corticocortical connections in the visual system: Structure and function. Physiol. Rev 75: 107-154.

    Google Scholar 

  • Schäfer AT, Roth A, Sakmann B (2000) Morphological correlates of BAC-firing threshold in model layer 5 pyramidal neurons. Forum of European Neuroscience Brighton. Eur. J. Neurosci. Supp. 167.08.

  • Schiller J, Major G, Koester HJ, Schiller Y (2000) NMDA spikes in basal dendrites of cortical pyramidal neurons. Nature 404: 285-289.

    Google Scholar 

  • Schiller J, Schiller Y, Stuart G, Sakmann B (1997) Calcium action potentials restricted to distal apical dendrites of rat neocortical pyramidal neurons. J. Physiol. (Lond.) 505: 605-616.

    Google Scholar 

  • Siegel M, Körding KP, König P (2000) Integrating bottom-up and top-down sensory processing by somato-dendritic interactions. J. Comput. Neurosci. 8: 161-173.

    Google Scholar 

  • Singer W, Gray CM (1995) Visual feature integration and the temporal correlation hypothesis. Ann. Rev. Neurosci. 18: 555-586.

    Google Scholar 

  • Softky W (1994) Submillisecond coincidence detection in active dendritic trees. Neuroscience 58: 13-41.

    Google Scholar 

  • Stone JV, Bray AJ (1995). A learning rule for extracting spatiotemporal invariances. Network: Comput. Neural Syst. 6: 429-436.

    Google Scholar 

  • Stuart GJ, Sakmann B (1994) Active propagation of somatic action potentials into neocortical pyramidal cell dendrites. Nature 367: 69-72.

    Google Scholar 

  • Stuart GJ, Schiller J, Sakmann B (1997) Action potential initiation and propagation in rat neocortical pyramidal neurons. J. Physiol. (Lond.) 505: 617-632.

    Google Scholar 

  • Tesauro G (1990) Neural models of classical conditioning: A theoretical viewpoint. In: SJ, Hanson CR Olson, eds. Connectionist Modeling and Brain Function. MIT Press: Cambridge, MA.

    Google Scholar 

  • Tsodyks MV, Markram H (1997) The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability. Proc. Natl. Acad. Sci. USA 94: 719-723.

    Google Scholar 

  • Werbos PJ (1974/1994) The Roots of Backpropagation. Wiley, New York (includes Ph.D. thesis of Werbos P from 1974).

  • Zador AM, Agmon-Snir H, Segev I (1995) The morphoelectrotonic transform: Agraphical approach to dendritic function. J. Neurosci. 15: 1669-1682.

    Google Scholar 

  • Zeki S, Shipp S (1988) The functional logic of cortical connections. Nature 335: 311-317.

    Google Scholar 

  • Zipser D, Andersen RA (1988) A backpropagation programmed network that simulates response properties of a subset of posterior parietal neurons. Nature 331: 679-684.

    Google Scholar 

  • Zipser D, Rumelhart DE (1990) Neurobiological significance of new learning models. In: E Schwartz, ed. Computational Neuroscience MIT Press: Cambridge, MA, pp. 192-200.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Körding, K.P., König, P. Supervised and Unsupervised Learning with Two Sites of Synaptic Integration. J Comput Neurosci 11, 207–215 (2001). https://doi.org/10.1023/A:1013776130161

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1013776130161

Navigation