Abstract
The problem of determining the nonlinear function (“blackbox”) which optimally associates (on given criteria) two sets of data is considered. The data are given as discrete, finite column vectors, forming two matricesX (“input”) andY (“output”) with the same numbers of columns and an arbitrary numbers of rows. An iteration method based on the concept of the generalized inverse of a matrix provides the polynomial mapping of degreek onX by whichY is retrieved in an optimal way in the least squares sense. The results can be applied to a wide class of problems since such polynomial mappings may approximate any continuous real function from the “input” space to the “output” space to any required degree of accuracy. Conditions under which the optimal estimate is linear are given. Linear transformations on the input key-vectors and analogies with the “whitening” approach are also discussed. Conditions of “stationarity” on the processes of whichX andY are assumed to represent a set of sample sequences can be easily introduced. The optimal linear estimate is given by a discrete counterpart of the Wiener-Hopf equation and, if the key-signals are noise-like, the holographic-like scheme of associative memory is obtained, as the optimal nonlinear estimator. The theory can be applied to the system identification problem. It is finally suggested that the results outlined here may be relevant to the construction of models of associative, distributed memory.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anderson, J.A.: A memory storage model utilizing spatial correlation functions. Kybernetik5, 113 (1968)
Anderson, J.A.: A simple neural network generating an interactive memory. Math. Biosci.14, 197 (1972)
Balakrishnan, A.V.: Identification of control systems from inputoutput data. International Federation of Automatic Control, Congress 1967
Barrett, J.F.: The use of functionals in the analysis of non-linear physical systems. J. Electr. Contr.15, 567 (1963)
Borsellino, A., Poggio, T.: Holographic aspects of temporal memory and optomotor response. Kybernetik10, 58 (1971)
Borsellino, A., Poggio, T.: Convolution and correlation algebras. Kybernetik13, 113 (1973)
Cooper, L.N.: A possible organization of animal memory and learning. Proc. of the Nobel Symposium on Collective Properties of Physical Systems. Lundquist, B. (ed). New York: Academic Press 1974
Dieudonné, J.: Foundations of modern analysis. New York: Academic Press 1969
Doob, J.L.: Stochastic processes. London: J. Wiley 1963
Gabor, D.: Associative holographic memories. IBM J. Res. Devel.13, 2 (1969)
Geiger, G., Poggio, T.: The orientation of flies towards visual pattern: on the search for the underlying functional interactions. Biol. Cybernetics (in press)
Heerden, P.J., van: Theory of optical information storage in solids. Appl. Optics2, 387 (1963)
Katzenelson, J., Gould, L.A.: The design of nonlinear filters and control systems. Part I. Information and Control5, 108 (1962)
Kohonen, T.: Correlation matrix memories. IEEE Trans. Comput. C21, 353–359 (1972)
Kohonen, T., Ruohonen, M.: Representation of associated data by matrix operators. IEEE Trans. on Comp. (1973)
Lee, Y.W., Schetzen, M.: Measurement of the Wiener kernels of a nonlinear system by cross-correlation. Int. J. Control2, 237 (1965)
Longuet-Higgins, H.C.: Holographic model of temporal recall. Nature (Lond.)217, 104 (1968)
Longuet-Higgins, H.C., Willshaw, D.J., Bunemann, O.P.: Theories of associative recall. Rev. Biophys.3, 223 (1970)
Marr, D.: A theory of cerebellar cortex. J. Phys. (Lond.)202, 437 (1969)
Marr, D.: A theory for cerebral neocortex. Proc. roy. Soc. B176, 161 (1970)
Penrose, R.: A generalized inverse for matrices. Proc. Cambridge Phil. Soc.52, 406 (1955)
Penrose, R.: On best approximate solutions of linear matrix equations. Proc. Cambridge Phil. Soc.,52, 17 (1956)
Pfaffelhuber, E.: Correlation memory models — a first approximation in a general learning scheme. Biol. Cybernetics (in press)
Poggio, T.: On holographic models of memory. Kybernetik12, 237 (1973)
Poggio, T.: In: Vecli, A. (ed.): Processing of visual information in flies. Proc. I. Symp. It. Soc. Biophys. Parma: Tipo-Lito 1974
Poggio, T.: A theory of nonlinear interactions in many inputs systems. In preparation (1975a)
Poggio, T.: On optimal associative black-boxes: theory and applications. In preparation (1975b)
Willshaw, D.J.: Models of distributed associative memory. Ph. D. Thesis. University of Edinburgh 1971
Willshaw, D.J.: A simple network capable of inductive generalization. Proc. roy. Soc. B182, 233 (1972)
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Poggio, T. On optimal nonlinear associative recall. Biol. Cybernetics 19, 201–209 (1975). https://doi.org/10.1007/BF02281970
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF02281970