Abstract
This report continues our research into the effectiveness of adaptive synaptogenesis in constructing feed-forward networks which perform good transformations on their inputs. Good transformations are characterized by the maintenance of input information and the removal of statistical dependence. Adaptive synaptogenesis stochastically builds and sculpts a synaptic connectivity in initially unconnected networks using two mechanisms. The first, synaptogenesis, creates new, excitatory, feed-forward connections. The second, associative modification, adjusts the strength of existing synapses. Our previous implementations of synaptogenesis only incorporated a postsynaptic regulatory process, receptivity to new innervation (Adelsberger-Mangan and Levy 1993a, b). In the present study, a presynaptic regulatory process, presynaptic avidity, which regulates the tendency of a presynaptic neuron to participate in a new synaptic connection as a function of its total synaptic weight, is incorporated into the synaptogenesis process. In addition, we investigate a third mechanism, selective synapse removal. This process removes synapses between neurons whose firing is poorly correlated. Networks that are constructed with the presynaptic regulatory process maintain more information and remove more statistical dependence than networks constructed with postsynaptic receptivity and associative modification alone. Selective synapse removal also improves network performance, but only when implemented in conjunction with the presynaptic regulatory process.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adelsberger-Mangan DM, Levy WB (1992) Information maintenance and statistical dependence reduction in simple neural networks. Biol Cybern 67:469–477
Adelsberger-Mangan DM, Levy WB (1993a) Character recognition using adaptively constructed feed-forward networks. Proc World Congress Neural Networks 11:423–426
Adelsberger-Mangan DM, Levy WB (1993b) Adaptive synaptogenesis constructs networks that maintain information and reduce statistical dependence. Biol Cybern 70:81–87
Atick JJ (1992) Could information theory provide an ecological theory of sensory processing? Network 3:213–251
Atick JJ, Redlich AN (1992) What does the retina know about natural scenes? Neural Comput 4:196–210
Barlow HB (1959) Sensory mechanisms, the reduction of redundancy, and intelligence. (National Physical Laboratory Symposium no. 10. The mechanisation of thought processes) Her Majesty's Stationary Office, London, pp 537–559
Barlow HB (1961a) The coding of sensory messages. In: Thorpe WH, Zangwill OL (eds) Current problems in animal behavior. Cambridge University Press, Cambridge, UK, pp 331–360
Barlow HB (1961b) Possible principles underlying the transformations of sensory messages. In: Rosenblith W (ed) Sensory communication. MIT Press, Cambridge, Ma. pp 217–234
Barlow HB (1985) Perception: what quantitative laws govern the acquisition of knowledge from the senses? In: Coen CW (ed) Functions of the brain. Clarendon Press, Oxford, pp 11–43
Barlow HB, Földiák P (1989) Adaptation and decorrelation in the cortex. In: Durbin RM, Miall C, Mitchison GJ (eds) The computing neuron. Addison-Wesley, Wokingham, pp 54–72
Buckingham J, Willshaw D (1993) On setting unit thresholds in an incompletely connected associative net. Network 4:441–459
DeSieno D (1988) Adding a conscience to competitive learning. IEEE Int Conf Neural Networks 1:117–124
Földiák P (1990) Forming sparse representations by local anti-hebbian learning. Biol Cybern 64:165–170
Grossberg S (1976) Adaptive pattern classification and universal recoding. 1. Parallel development and coding of neural feature detectors. Biol Cybern 23:121–134
Hateren van J (1992) A theory of maximizing sensory information. Biol Cybern 68:23–29
Levy WB (1985) An information/computation theory of hippocampal function. Soc Neurosci Abst 11:493
Levy WB (1989) A computational approach to hippocampal function. In: Hawkins RD, Bower GH (eds) Computational models of learning in simple neural systems. Academic, San Diego, pp 243–305
Levy WB, Colbert CM (1991) Adaptive synaptogenesis can complement associative modification/depression. In: Commons ML (ed) Neural network models of conditioning: quantitative analyses of behavior, 13. Lawrence Erlbaum, Hillsdale, pp 53–68
Levy WB, Desmond NL (1985) The rules of elemental synaptic plasticity. In: Levy WB, Anderson JA, Lehmkuhle S (eds) Synaptic modification, neuron selectivity, and nervous system organization. Lawrence Erlbaum, Hillsdale, pp 105–121
Linsker R (1988) Self-organization in a perceptual network. Computer 21:105–117
Linsker R (1990) Perceptual neural organization: some approaches based on network models and information theory. Annu Rev Neurosci 13:257–281
Linsker R (1992) Local synaptic learning rules suffice to maximize mutual information in a linear network. Neural Comput. 4:672–683
Oja E (1989) Neural networks, principle components, and subspaces. Int J Neural Syst 1:61–68
Richards DSP, Levy WB (1990) Optimum preprocessing networks and a data processing theorem. Proc Int J Conf Neural Networks 1:19–22
Rolls ET (1989) Information processing in the taste system of primates. J Exp Biol 146:141–164
Rubner J, Tavan P (1989) A self-organizing network for principle component analysis. Europhys Lett 10:693–698
Sanger TD (1989a) Optimal unsupervised learning in a single-layer linear feed forward neural network. Neural Networks 2:459–473
Sanger TD (1989b) An optimality principle for unsupervised learning. in: Touretzky DS (ed) Advances in Neural Information Processing Systems, I. Morgan Kaufmann, San Mateo, pp 11–19
Watanabe S (1960) Information theoretical analysis of multivariate correlation. IBM J Res Dev 4:66
Watanabe S (1969) Knowing and guessing: a quantitative study of inference and information. Wiley, New York
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Adelsberger-Mangan, D.M., Levy, W.B. The influence of limited presynaptic growth and synapse removal on adaptive synaptogenesis. Biol. Cybern. 71, 461–468 (1994). https://doi.org/10.1007/BF00198922
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF00198922