Abstract
A review of recent patent applications indicates that neural networks using Hamming-type algorithms with minimum-mismatch selection provide an optimal combination of implementational simplicity, information storage capacity and signal-noise characteristics. These networks can be adapted to implement Bayes' rule, by setting link gains to the negative logarithm of conditional or a priori probabilities. Where probability distributions and noise are not uniform or random, the performance of Bayesian classifiers may be significantly better than that of the corresponding Hamming network on the same vector set. We demonstrate this for the noisy digit classification task. We also generate biologically plausible curvature detectors for character recognition and compare the performances of Bayesian and Hamming networks at classifying the resultant vectors. Preliminary results suggest that Hamming networks may provide good approximations to the Bayes optimum for sparse natural vector sets under some conditions.
supported by a grant from MoD R.A.R.D.E.
supported by a grant from DGICYT
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
10 References
Baum E.B., Moody J., Wilczec F., 1988 Biological Cybernetics, 59, 217–228.
Blake A. 1983 The least disturbance principle and weak constraints. Pattern Recognition Letters, 1, 393–399.
Burton G.J., Haig N.D., Moorhead I.R. 1986 A Self-similar stack model for human and machine vision. Biological Cybernetics, 53, 397–403.
Denker J.S. 1988 Highly parallel computation network employing a binary-valued T matrix and single output amplifiers. United States Patent Number 4,737,929 (filed Apr. 14, 1986).
Denker J.S. 1986 Neural network models of learning and adaptation. Physica D, vol. 22, 216–243.
Devijver P., Kittler J., 1982 Pattern Recognition: a Statistical Approach. (Prentice-Hall:N.J.).
Dobson V.G., 1975 Pattern learning and the control of behaviour in inhibitory neural network hierarchies. Perception 4, 35–50.
Dobson V.G., 1988 Signal Processing. International Patent Application. WO 88/01079 (filed Aug. 8, 1986)
Dobson V.G., 1987 Superior accuracy of Decrementing over Incrementing associative networks when operating in initially random connectivities. Journal of Intelligent Systems. 1, 43–78.
Dobson V.G., 1989 Decrementing Networks, In L. Personnaz, G. Dreyfus, Eds. Neural Networks: from Models to Applications (IDSET:Paris) 316–325.
Dobson V.G., Salinas R.M. Decrementing Hamming and Bayesian neural networks. Journal of Intelligent Systems. In press.
Duda R.O., Hart P.E., 1973 Pattern Classification and Scene Analysis. (J. Wiley:New York).
Gardner E. 1988 The space of interactions in neural network models. Journal of physiology, A, Mathematics
General 21 271–279. Goldstein M. Dillon W.R. 1978 Discrete Discriminant Analysis. (J.Wiley: New York)
Graf H.P., Jackel L.D., Hubbard W.E., 1988 VLSI implementation of a neural network model. Computer: Institute of Electrical and Electronic Engineers, March, 41–49.
Hopfield J.J., 1982 Neural networks and physical systems with emergent collective computational abilities. Proceedings of National Academy of Science, U.S.A. 79, 2554–2558.
Hopfield J.J., 1984 Neurons with graded responses have collective computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. U.S.A. 81, 3088.
Hopfield J.J., Tank D.W., 1986 Computing with neural circuits: a model. Science 233, 625.
Hopfield J.J., 1987 Electronic network for collective decision based on large number of connections between signals. United States Patent Number 4,660,166 (filed Jan. 22, 1985).
Kohonen T., Barna, G., Chrisley R., 1988 Statistical pattern recognition with neural networks: benchmarking studies. Proceedings of 2nd International Conference on Neural networks; Institute of Electrical and Electronic Engineers, San Diego I 61–69.
Kononenko I, 1989 Bayesian Neural Networks. Biological Cybernetics 61, 361–370.
Lansner A., Ekeberg O., 1985 Reliability and speed of recall in an associative network. Institute of Electrical and Electronic Engineers: Transactions on Pattern Analysis and Machine Intelligence 7(4) 490–498.
Linsker R. 1988 Self-organization in a perceptual network. Computer: Institute of Electrical and Electronic Engineers, March, 105–117.
Lippmann R.P., 1987 An introduction to computing with neural nets. Institute of Electrical and Electronic Engineers ASSP Magazine. April, 4–22.
Moopenn A., Thakoor A.P., 1989 Hybrid analog-digital associative neural networks. United States Patent Number 4,807,168 (filed Jun. 10, 1987).
Moopenn A., Thakoor A.P., Duong T. 1988 A neural network for Euclidean distance minimisation. Proceedings of 2nd International Conference on Neural networks; Institute of Electrical and Electronic Engineers, San Diego I, 349–356.
Nadal J. Toulouse G 1990 Information storage in sparsely coded memory nets. Network 1 61–74.
Tarassenko L., Seifert B.G., Toombs J.N., 1989 Neural network architectures for associative memory Proceedings 1st. International Conference on Analogue Neural Networks. Institute of Electrical Engineers Publications. 313, 17–22.
Thorpe S.J., Imbert M. 1989 Biological constraints on connectionist modelling. In Pfeiffer, R. Connectionism in Perspective. (J.Wiley: New York) 316–325. Willshaw D.J., Buneman O.P., Longuet-Higgins C.H., 1969 Non-holographic associative memory. Nature, 222, 960–962.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1991 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dobson, V.G., Salinas, J.M. (1991). Decrementing hamming and Bayesian neural networks: Analog implementations and relative performance. In: Prieto, A. (eds) Artificial Neural Networks. IWANN 1991. Lecture Notes in Computer Science, vol 540. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0035893
Download citation
DOI: https://doi.org/10.1007/BFb0035893
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-54537-8
Online ISBN: 978-3-540-38460-1
eBook Packages: Springer Book Archive