Abstract
Rationality necessitates that the neuron must base its decisions on probability measures established on the interrogative proposition q which essentially defines the neuron as an observer and which is established during learning. This requirement is induced by the fact that probability is the only measure of degree of plausible belief which meets all logical consistency requirements. Similarly, the establishment of a probability measure by a rational neuron must entail the usage of a maximized entropy criterion since this is the only logically consistent means of doing so as based on observable information. A maximum entropy (ME) formulation can be shown to provide the basic functional form of the model neuron including synaptic weights and a sigmoidal transfer characteristic. However, this formulation requires the specification of linear constraints which are unavailable. Alternatively, a maximum mutual information (MMI) formulation is shown to be fully constrained in this regard and can make exclusive use of locally available information. Solutions take the form of the Hopfield neuron model with a requirement for Hebbian learning.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Francisco J. Varela, Principles of Biological Autonomy, North Holland, 1979.
Solomon Kullback, Information theory and statistics, Wiley, 1959 and Dover, 1968.
George Spencer-Brown, Laws of Form, E. P. Dutton, New York 1979
R.T. Cox, The algebra of probable inference, The Johns Hopkins Press, Baltimore, 1961.
Hopfield, J. J., “Neural networks and physical systems with emergent collective computational abilities,” Proc. Nat. Acad. Sci. USA, 79, 2554–2558, 1982.
R. L. Fry, “Observer-participant models of neural processing,” IEEE Trans. Neural Networks, July, 1995.
E. Oja, “A simplified neuron model as a principal component analyzer,” J. of Math. Biol. 15, 267–273., 1982
William B. Levy, Costa M. Colbert, and Nancy L. Desmond, “Elemental adaptive processes of neurons and synapses: A statistical/computational perspective,” in Neuroscience and connectionists theory, pp 187–235, 1990.
William B. Levy, “Maximum entropy prediction in neural networks,” in proc. International Joint Conference on Neural Networks, 1990.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1996 Springer Science+Business Media Dordrecht
About this paper
Cite this paper
Fry, R.L. (1996). Rational Neural Models Based on Information Theory. In: Hanson, K.M., Silver, R.N. (eds) Maximum Entropy and Bayesian Methods. Fundamental Theories of Physics, vol 79. Springer, Dordrecht. https://doi.org/10.1007/978-94-011-5430-7_41
Download citation
DOI: https://doi.org/10.1007/978-94-011-5430-7_41
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-010-6284-8
Online ISBN: 978-94-011-5430-7
eBook Packages: Springer Book Archive