Skip to main content

“Neural” computation of decisions in optimization problems

Abstract

Highly-interconnected networks of nonlinear analog neurons are shown to be extremely effective in computing. The networks can rapidly provide a collectively-computed solution (a digital output) to a problem on the basis of analog input information. The problems to be solved must be formulated in terms of desired optima, often subject to constraints. The general principles involved in constructing networks to solve specific problems are discussed. Results of computer simulations of a network designed to solve a difficult but well-defined optimization problem-the Traveling-Salesman Problem-are presented and used to illustrate the computational power of the networks. Good solutions to this problem are collectively computed within an elapsed time of only a few neural time constants. The effectiveness of the computation involves both the nonlinear analog response of the neurons and the large connectivity among them. Dedicated networks of biological or microelectronic neurons could provide the computational capabilities described for a wide class of problems having combinatorial complexity. The power and speed naturally displayed by such collective networks may contribute to the effectiveness of biological information processing.

This is a preview of subscription content, access via your institution.

References

  1. Anderson, P.W.: Ch. 2. In: Basic notions of condensed matter physics. Menlo Park: Benjamin Cummings 1984

    Google Scholar 

  2. Arbib, M.A.: Artificial intelligence and brain theory: unities and diversities. Ann. Biomed. Eng. 3, 238–274 (1975)

    Google Scholar 

  3. Ballard, D.H.: Cortical connections and parallel processing: structure and function. Behav. Brain Sci. (1985, in press)

  4. Ballard, D.H., Hinton, G.E., Sejnowski, T.J.: Parallel visual computation. Nature 306, 21–26 (1983)

    Google Scholar 

  5. Feldman, J.A., Ballard, D.H.: Connectionist models and their properties. Cog. Sci. 6, 205–254 (1982)

    Google Scholar 

  6. Garey, M.R., Johnon, D.S.: Computers and intractability. New York: W.H. Freeman 1979

    Google Scholar 

  7. Gelperin, A.E., Hopfield, J.J., Tank, D.W.: The logic of Limax learning. In: Model neural networks and behavior. Selverston, A., ed.: New York: Plenum Press 1985

    Google Scholar 

  8. Goldman-Rakic, P.S.: Modular organization of the prefrontal cortex. Trends Neurosci, 7, 419–424 (1984)

    Google Scholar 

  9. Gross, D.J., Mezard, M.: The simplest spin glass. Nucl. Phys. FS B 240, 431–452 (1984)

    Google Scholar 

  10. Hinton, G.E., Sejnowski, T.J.: Optimal perceptual inference. In: Proc. IEEE Comp. Soc. Conf. Comp. Vision & Pattern Recog., pp. 448–453, Washington, D.C., June 1983

  11. Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982)

    Google Scholar 

  12. Hopfield, J.J.: Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. USA 81, 3088–3092 (1984)

    Google Scholar 

  13. Huskey, H.D., Korn, G.A.: Computer Handbook, pp. 1–4. New York: McGraw Hill 1962

    Google Scholar 

  14. Ikeuchi, K., Horn, B.K.P.: Numerical shape from shading and occluding boundaries. Artif. Intell. 17, 141–184 (1981)

    Google Scholar 

  15. Jackson, W.A.: Analog computation, pp. 5–8. New York: McGraw Hill 1960

    Google Scholar 

  16. Julesz, B.: In: Foundations of cycleopean vision. Chicago: University of Chicago Press 1971

    Google Scholar 

  17. Julesz, B.: Textons, the elements of texture perception, and their interactions. Nature 290, 91–97 (1981)

    Google Scholar 

  18. Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220, 671–680 (1983)

    Google Scholar 

  19. Koch, C., Poggio, T., Torre, V.: Nonlinear interactions in a dendritic tree: Localization, timing, and role in information processing. Proc. Natl. Acad. Sci. USA 80, 2799 (1983)

    Google Scholar 

  20. Lawler, E.L., Lenstra, A., Rinnooy Kan, H. G., Eds.: The traveling salesman problem. New York: John Wiley (in press)

  21. Lin, S., Kernighan, B.W.: An effective heuristic algorithm for the traveling-salesman problem. Oper. Res. 21, 498–516 (1973)

    Google Scholar 

  22. Marr, D.: Vision. New York: W. H. Freeman 1982

    Google Scholar 

  23. Mead, C., Conway, L.: Introduction to VLSI systems, pp. 12,265. Reading, MA: Addison-Wesley 1980

    Google Scholar 

  24. Poggio, T., Koch, C.: An analog model of computation for the illposed problems of early vision. M.I.T. Artif. Intell. Lab. A.I. Memo No. 783, 1984

  25. Poggio, T., Torre, V.: Ill-posed problems in early vision. Proc. Roy. B. (1985, in press)

  26. Sebestyn, G.S.: Decision-making processes in pattern recognition. New York: McMillan 1962

    Google Scholar 

  27. Shepherd, G.M.: Microcircuits in the nervous system. Sci. Am. 238, 92–103 (1978)

    Google Scholar 

  28. Terzopoulos, D.: Multilevel reconstruction of visual surfaces: variational principles and finite element representations. In: Multi-resolution image processing and analysis. Rosenfeld, A., ed.. Berlin, Heidelberg, New York: Springer 1984

    Google Scholar 

  29. Thouless, D.J., Anderson, P.W., Palmer, R.G.: Solution of “solvable model of a spin glass”. Phil. Mag. 35, 593–601 (1977)

    Google Scholar 

  30. Tomovic, R., Karplus, W.J.: High speed analog computers, p. 4. New York: Wiley 1962

    Google Scholar 

  31. Wannier, G.H.: Statistical physics, p. 330ff. New York: Wiley 1966

    Google Scholar 

Download references

Author information

Affiliations

Authors

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Hopfield, J.J., Tank, D.W. “Neural” computation of decisions in optimization problems. Biol. Cybern. 52, 141–152 (1985). https://doi.org/10.1007/BF00339943

Download citation

Keywords

  • Time Constant
  • General Principle
  • Specific Problem
  • Computational Power
  • Wide Class