Abstract
We study the influence of a variable neuronal threshold on fixed points and convergence rates of an associative neural network in the presence of noise. We allow a random distribution in the activity levels of the patterns stored, and a modification to the standard Hebbian learning rule is proposed for this purpose. There is a threshold at which the retrieval ability, including the average final overlap and the convergence rate, is optimized for patterns with a particular activity level at a given noise level. This type of selective attention to one class of patterns with a certain activity level may be obtained at the cost of reducing the retrieval ability of the network for patterns with different activity levels. The effects of a constant threshold independent of noise, time, and pattern are discussed. For high-(low-) activity patterns, the average final overlap is shown to be increased at high noise levels and decreased at low noise levels by a negative (positive) constant threshold, whereas a positive (negative) threshold always reduces the final average overlap. When the magnitude of the constant threshold exceeds a critical value, there is no retrieval. Rates of convergence towards the stored pattern with negative (positive) thresholds are greater than those with positive (negative) thresholds. These results are related to (de)sensitization and anesthesia. For certain threshold values and patterns with certain activity levels, hysteresis appears in the plot of the average final overlap versus the noise level, even for first order interactions. We make the analogy between the pattern-dependent neuronal threshold proposed in the present paper and the “task-related” modulation in neuronal excitability determined by cognitive factors, such as the attentional state of a higher animal. A constant threshold is associated with overall changes in neuronal excitability caused, e.g., by various drugs and physical injuries. Neurophysiological evidence of a dynamically variable neuronal threshold, such as accommodation and potentiation, is presented.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abelles M (1982) Local cortical circuits. Springer, New York Berlin Heidelberg, p21
Alberts B, Bray D, Lewis J, Raff M, Roberts K, Watson JD (1989) Molecular biology of the cell, 2nd edn. Garland, New York
Amit DJ, Gutfreund H, Sompolinsky H (1985) Spin-glass models of neural networks. Phys Rev A32:1007–1018
Amit DJ, Gutfreund H, Sompolinsky H (1987a) Information storage in neural networks with low levels of activity. Phys Rev A35:2293–2303
Amit DJ, Gutfreund H, Sompolinsky H (1987b) Statistical mechanics of neural networks near satuation. Ann Phys (NY) 173:30–67
Bradley K, Somjen GG (1961) Accommodation in motoneurons of the rat and the cat. J Physiol 156:75–92
Buhmann J, Divko R, Schulten K (1989) Associate memory with high information content. Phys Rev A39:2689–2692
Bushnell MC, Duncan GH, Dubner R, He LF (1984) Activity of trigeminothalamic neurons in medullary dorsal horn of awake monkeys trained in a thermal discrimination task. J Neurophysiol 52:170–187
Bushnell MC, Duncan GH, Lund JP (1987) Gating of sensory transmission in the trigeminal system. In: Wise SP (ed) Higher brain functions. Wiley, New York
Carpenter RHS (1984) Neurophysiology. University Park Press, Baltimore, Md
Cortes C, Krogh A, Hertz JA (1987) Hierarchical associative networks. J Phys A20:4449–4455
Crick F (1989) The recent excitement about neural networks. Nature 337:129–132
Derrida B, Gardner E, Zippelius A (1987) An exactly solvable asymmetric neural network model. Europhys Lett 4: 167–173
Desmedt JE (ed) (1977) Attention, voluntary contraction and eventrelated cerebral potentials. Karger, New York; (1979) Cognitive components in cerebral event-related potentials and selective attention. Karger, New York
Domany E (1988) Neural networks: a biased overview. J Statist Phys 51:743–775
Fabre P (1927) L'excitation neuro-musculaire par les courants progressifs chez l'homme. C R Acad Sci (Paris) 184:699–701
Gardner E (1988) The space of interactions in neural network models. J Phys A21: 257–270
Grossberg S (1970) Neural pattern discrimination. J Theor Biol 27:291–337
Gutfreund H (1988) Neural networks with hierarchically correlated patterns. Phys Rev A37:570–577
Guyton AC (1981) Basic human neurophysiology. Saunders, Philadelphia
Hebb DO (1949) The organization of behavior. Wiley, New York, p 44
Hill AV (1936) Excitation and accommodation in nerve. Proc R Soc (London) Ser B 119:305–355
Hodgkin AL, Huxley AF (1952) A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol 117:500–544
Hoffmann GW (1986) A neural network model based on the analogy with the immune system. J Theor Biol 122:33–67
Hoffmann GW, Benson MW, Bree GM, Kinahan P (1986) A teachable neural network model based on an unorthodox neuron. Physica 22D:233–246
Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci USA 79:2554–2558
Hopfield JJ (1984) Neurons with graded response have collective computational properties like those of two-state neurons. Proc Natl Acad Sci USA 81:3088–3092
Keeler JD, Pichler EE, Ross J (1989) Noise in neural networks: Thresholds, hysteresis, and neuromodulation of signal-to-noise. Proc Natl Acad Sci USA 86:1712–1716
Kinzel W (1985) Learning and pattern recognition in spin glass models. Z Phys B-Condensed matter 60:205–213
Kohonen T (1988) Self organization and associative memory. Springer Series in Information Science, vol 8, 2nd edn. Springer, Berlin Heidelberg New York
Krogh A, Hertz JA (1988) Mean-field analysis of hierarchical assorciative networks with ‘magnetisation’. J Phys A21: 2211–2224
Kuffler S, Nicholls JG, Martin AR (1984) From neuron to brain, 2nd edn. Sinauer, Sunderland, Mass
Leventhal H, Brown D, Shacham S, Engquist G (1979) Effects of preparatory information about sensations, threat of pain, and attention on cold pressor distress. J Personal Social Psychol 37:688–714
Little WA (1974) The existence of persistent states in the brain. Math Biosci 19:101–120
McCulloch WS, Pitts W, Bull (1943) A logical calculus of the ideas immanent in nervous activity. Math Biophys 5:115–133
Noest AJ (1988) Phasor neural networks. In: Anderson DZ (ed) Neural information processing system. American Institute of Physics, New York, pp 584–591
Parga N, Virasoro MA (1986) The ultrametric organization of memories in a neural network. J Phys (Paris) 47:1857–1864
Peretto P, Niez JJ (1986) Stochastic dynamics of neural works. IEEE Trans SMC-16:73–83
Sompolinsky H (1988) Statistical mechanics of neural works. Phys Today 41:70–80
Wang L, Ross J (1990a) Synchronous neural networks of non-linear threshold elements with hysteresis. Proc Natl Acad Sci (USA) 87:988–992
Wang L, Ross J (1990b) Interactions of neural networks: models for distraction and concentration. Proc Natl Acad Sci (USA) 87:7110–7114.
Willis WD Jr, Grossman RG (1977) Medical neurobiology, 2nd edn. Mosby, St. Louis
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Wang, L., Ross, J. Variable threshold as a model for selective attention, (de)sensitization, and anesthesia in associative neural networks. Biol. Cybern. 64, 231–241 (1991). https://doi.org/10.1007/BF00201984
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF00201984