Skip to main content
SpringerLink
Log in

Linear function neurons: Structure and training

  • Published:
Biological Cybernetics Aims and scope Submit manuscript

Abstract

Three different representations for a thresholded linear equation are developed. For binary input they are shown to be representationally equivalent though their training characteristics differ.

A training algorithm for linear equations is discussed. The similarities between its simplest mathematical representation (perceptron training), a formal model of animal learning (Rescorla-Wagner learning), and one mechanism of neural learning (Aplysia gill withdrawal) are pointed out. For d input features, perceptron training is shown to have a lower bound of 2d and an upper bound of d d adjusts. It is possible that the true upper bound is 4d, though this has not been proved. Average performance is shown to have a lower bound of 1.4d. Learning time is shown to increase linearly with the number of irrelevant or replicated features. The (X of N) function (a subset of linearly separable functions containing OR and AND) is shown to be learnable in d 3 time.

A method of utilizing conditional probability to accelerate learning is proposed. This reduces the observed growth rate from 4d to the theoretical minimum (for the unmodified version) of 2d. A different version reduces the growth rate to about 1.7d. The linear effect of irrelevant features can also be eliminated. Whether such an approach can be made provably convergent is not known.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abraham WC, Goddard GV (1984) Functions of afferent coactivity in long-term potentiation. In: Lynch G, McGaugh JL, Weinberger NM (eds) Neurophysiology of learning and memory. Guilford Press, New York

    Google Scholar 

  • Adrain ED (1946) The physical background of perception. Clarendon Press, Oxford

    Google Scholar 

  • Agmon S (1954) The relaxation method for linear inequalities. Can J Math 6:382–392

    Google Scholar 

  • Amari S (1977a) Neural theory of association and concept formation. Biol Cybern 26:175–185

    Google Scholar 

  • Amari S (1977b) Dynamics of pattern formation in lateral-inhibition type neural fields. Biol Cybern 27:77–78

    Google Scholar 

  • Barlow HB (1972) Single units and sensation: a neuron doctrine for perceptual psychology. Perception 1:371–394

    Google Scholar 

  • Barrionuevo G, Brown TH (1983) Associative long-term synaptic potentiation in hippocampal slices. Proc Natl Acad Sci USA 80:7347–7351

    Google Scholar 

  • Barto AG, Sutton RS (1981) Landmark learning: an illustration of associative search. Biol Cybern 42:1–8

    Google Scholar 

  • Barto AG, Sutton RS, Brouwer PS (1981) Associative search network: a reinforcement learning associative memory. Biol Cybern 40:201–211

    Google Scholar 

  • Barto AG, Sutton RS, Anderson CW (1982) Neuron-like adaptive elements that can solve difficult learning control problems. Comp. and Info. Sci. Dept. U. of Mass. Amherst, Mass. Tech report 82-20

  • Belnap ND Jr (1977) A useful four-valued logic. In: Dunn JM, Epstein G, (eds) Modern uses of multiple-valued logic. Reidel, Boston

    Google Scholar 

  • Bindra D (1976) A theory of intelligent behavior. Wiley, New York

    Google Scholar 

  • Bourne LE (1970) Knowing and using concepts. Psychol Rev 77:546–556

    Google Scholar 

  • Bourne LE, Restle F (1959) Mathematical theory of concept identification. Psychol Rev 66:278–296

    Google Scholar 

  • Bulgarella RG, Archer EJ (1962) Concept identification of auditory stimuli as a function of amount of relevant and irrelevant information. J Exp Psychol 63:254–257

    Google Scholar 

  • Cameron SH (1960) An estimate of the complexity requisite in a universal decision network. Bionics Symposium, WADD Report 60-600, pp 197–212

  • Carew TJ, Hawkins RD, Kandel ER (1983) Differential classical conditioning of a defensive withdrawal reflex in Aplysia Californica. Science 219:397–400

    Google Scholar 

  • Cohen B, Murphy GL (1984) Models of concepts. Cog Sci 8:27–58

    Google Scholar 

  • Crow T (1984) Cellular neurophysiological and behavioral studies of learning in mollusks. In: Squire LR, Bulters N (eds) Neuropsychology of Memory. Guilford Press, New York

    Google Scholar 

  • Dickinson A (1980) Contemporary animal learning theory. Cambridge University Press, New York

    Google Scholar 

  • Doyle J (1979) A truth maintenance system. Artifi Intell 12:231–272

    Google Scholar 

  • Duda R, Hart P (1973) Pattern classification and scene analysis. John Wiley, New York

    Google Scholar 

  • Estes WK (1976) The cognitive side of probability learning. Psychol Rev 83:37–64

    Google Scholar 

  • Gelperin A, Culligan N (1984) In vitro expression of in vitro learning by an isolated molluscan. CNS. Brain Res 304:207–213

    Google Scholar 

  • Gibbon J (1981) The contingency problem in autoshaping. In: Locurto CM, Terrace HS, Gibbon J (eds) Autoshaping and conditioning theory. Academic Press, New York

    Google Scholar 

  • Gibbon J, Berryman R, Thompson RL (1974) Contingency spaces and measure in classical and instrumental conditioning. J Exp Anal Behav 21:585–605

    Google Scholar 

  • Gold PE (1984) Memory modulation: neurobiological controls In: Lynch G, McGaugh JL, Weinberger NM (eds) Neurophysiology of learning and memory. Guilford Press, New York

    Google Scholar 

  • Hampson SE (1983) Neural model of adaptive behavior. PhD dissertation, Information and Computer Science Dept., Univ. of Calif., Irvine. Tech. Rep. 213

  • Hampson SE, Kibler D (1983) A Boolean complete neural model of adaptive behavior. Biol Cybern 49:9–19

    Google Scholar 

  • Hawkins RD (1983) Cellular neurophysiological studies of learning. In: Deutsch JA (ed) The physiological basis of memory. Academic Press, New York

    Google Scholar 

  • Hawkins RD, Kandel ER (1984) Is there a cell-biological alphabet for simple forms of learning? Psychol Rev 91:375–391

    Google Scholar 

  • Hinton GE, Anderson JA (eds) (1981) Parallel models of associative memory. Lawrence Erlbaum, Hillsdale

    Google Scholar 

  • Hu Sze-Tsen (1965) Threshold logic. University of California Press, Berkeley

    Google Scholar 

  • Hunt E, Martin J, Store P (1966) Experiments in induction. Academic Press, New York

    Google Scholar 

  • Kandel ER (1976) Cellular basis of behavior. Freeman, San Francisco

    Google Scholar 

  • Kandel ER (1979a) Behavioral biology of Aplysia. Freeman, San Francisco

    Google Scholar 

  • Kandel ER (1979b) Small systems of neurons. Sci Am 241:66–76

    Google Scholar 

  • Kandel ER, Schwartz J (eds) (1981) Principles of neural science. Elsevier North-Holland, New York

    Google Scholar 

  • Kandel ER, Schwartz J (1982) Molecular biology of learning: Modulation of transmitter release. Science 218:433–443

    Google Scholar 

  • Kent EW (1981) The brains of men and machines. BYTE/McGraw-Hill, Peterborough

    Google Scholar 

  • Klopf AH (1982) The hedonistic neuron. Hemisphere Publishing Corp. Washington

    Google Scholar 

  • Kuffler SW, Nicholls JG, Martin AR (1984) From neuron to brain. Sinaver Sunderland

  • Lolordo VM (1979) Classical conditioning: contingency and contiguity. In: Bitterman ME, Lolordo VM, Overmier JB, Rashotte ME (eds) Animal learning. Plenum Press, New York

    Google Scholar 

  • Mackintosh NJ (1975) A theory of attention: variations in the associability of stimuli with reinforcement. Psychol Rev 82:276–298

    Google Scholar 

  • Mackintosh NJ (1983) Conditioning and associative learning. Oxford Univ. Press, New York

    Google Scholar 

  • McCawley JD (1981) Everything that linguists have always wanted to know about logic. Univ. of Chicago Press, Chicago

    Google Scholar 

  • McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5:115–133

    Google Scholar 

  • McEntee WJ, Mair RG (1984) Some behavioral consequences of neurochemical deficits in Korsakoff psychosis. In: Squire LR, Butters N (eds) Neuropsychology of memory. Guilford Press, New York

    Google Scholar 

  • McNaughton BL (1983) Activity dependent modulation of hippocampal synaptic efficacy: some implications for memory processes. In: Seifert W (ed) Neurobiology of the hippocampus. Academic Press, New York

    Google Scholar 

  • Medin DL, Schwanenflugel PJ (1981) Linear separability in classification learning. J Exp Psych Hum Learn Mem 7:353–368

    Google Scholar 

  • Medin DL, Smith EE (1984) Concepts and concept formation. Annu Rev Psychol 35:113–138

    Google Scholar 

  • Mervis CB, Rosch E (1981) Categorization of natural objects. Annu Rev Psychol 32:89–115

    Google Scholar 

  • Minsky M, Papert S (1972) Perceptrons. MIT Press, Cambridge

    Google Scholar 

  • Muroga S (1965) Lower bounds of the number of threshold functions and a maximum weight. IEEE Trans. Electronic Computers 14:136–148

    Google Scholar 

  • Muroga S (1971) Threshold logic and its applications. Wiley, New York

    Google Scholar 

  • Muroga S, Toda L (1966) Lower bounds on the number of threshold functions. IEEE Trans Electron Comput 15:805–806

    Google Scholar 

  • Neisser U, Weene P (1962) Hierarchies in concept attainment. J Exp Psychol 64:640–645

    Google Scholar 

  • Nilsson NJ (1965) Learning machines. McGraw-Hill, New York

    Google Scholar 

  • Pearce JM, Hall G (1980) A model of Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev 87:532–552

    Google Scholar 

  • Rescorla RA (1967) Pavlovian conditioning and its proper control procedures. Psychol. Rev. 74:71–80

    Google Scholar 

  • Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning. In: Black AH, Prokasy WF (eds) Classical conditioning. II. Appleton-Century-Crofts, New York

    Google Scholar 

  • Rogawski MA, Aghajanian GK (1980) Modulation of lateral geniculate neuron excitability by noradrenaline microiontophoresis or locus ceruleus stimulation. Nature 287:731–734

    Google Scholar 

  • Rosch E (1978) Principles of categorization. In: Rosch E, Lloyd BB (eds) Cognition and categorization. Lawrence Erlbaum, Hillsdale

    Google Scholar 

  • Sahley C, Rudy JW, Gelperin A (1981) An analysis of associative learning in a terrestrial mollusc. J Comp Physiol 144:1–8

    Google Scholar 

  • Sejnowski TJ (1981) Skeleton filters in the brain. In: Hinton GE, Anderson JA (eds) Parallel models of associative memory. Lawrence Erlbaum, Hillsdale

    Google Scholar 

  • Shapiro SC (1979) The SNePs semantic network processing system. In: Findler NV (ed) Associative networks Academic Press, New York

    Google Scholar 

  • Shepherd GM (1983) Neurobiology. Oxford Univ. Press, New York

    Google Scholar 

  • Shortliffe EH, Buchanan BG (1975) A model of inexact reasoning in medicine. Math Biosci 23:351–379

    Google Scholar 

  • Smith EE, Medin DL (1981) Categories and concepts. Harvard Univ. Press, Cambridge

    Google Scholar 

  • Staddon JER (1983) Adaptive behavior and learning. Cambridge Univ. Press, New York

    Google Scholar 

  • Sutton RS, Barto AG (1981) Toward a modern theory of adaptive networks: expectation and prediction. Psychol Rev 88:135–170

    Google Scholar 

  • Thompson RF (1985) The brain. Freeman, New York

    Google Scholar 

  • Tucker DM, Williamson PA (1984) Asymmetric neural control systems in human self-regulation. Psychol. Rev. 91:185–215

    Google Scholar 

  • Tversky A (1977) Features of similarity. Psychol Rev 84:327–352

    Google Scholar 

  • Uttley AM (1979) Information transmission in the nervous system. Academic Press, New York

    Google Scholar 

  • Widrow G, Hoff ME (1960) Adaptive switching circuits. IRE Western Electronic Show and Convention, Convention Record Part 4, 96–104

    Google Scholar 

  • Widrow B, Gupta NK, Maitra S (1973) Punish/reward: learning with a critic in adaptive threshold systems. IEEE Trans Syst Man Cybern 3:455–465

    Google Scholar 

  • Winder RO (1961) Single stage threshold logic. Switching circuit theory and logical design, pp 321–332

  • Winder RO (1962) Threshold logic. Doctoral dissertation, Mathematics Department, Princeton University

  • Woodward DJ, Moises HC, Waterhouse BD, Hoffer BJ, Freeman R (1979) Modulatory actions of norepinephrine in the central nervous system. Fed Proc 38:2109–2116

    Google Scholar 

  • Woolridge DE (1979) Sensory processing in the brain. Wiley, New York

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information and Computer Science, University of California, Irvine, 92717, Irvine, CA, USA

    S. E. Hampson & D. J. Volper

Authors
  1. S. E. Hampson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. D. J. Volper
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hampson, S.E., Volper, D.J. Linear function neurons: Structure and training. Biol. Cybern. 53, 203–217 (1986). https://doi.org/10.1007/BF00336991

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00336991

Keywords

Search

Navigation