Skip to main content
SpringerLink
Log in

Computational complexity, learning rules and storage capacities: A Monte Carlo study for the binary perceptron

  • Original Contributions
  • Published:
Zeitschrift für Physik B Condensed Matter

Abstract

We examine the storage capacity for the binary perceptron using simulated annealing. In particular, we clarify the connection between the computational complexity of learning algorithms and the attained storage capacity. From finite-size studies we obtain a critical storage capacity,α c (κ)=0.8331±0.0016, in good agreement with the replica analysis of Krauth and Mézard. However, we demonstrate that a polynomial time cooling schedule yields a vanishing storage capacity in the thermodynamic limit as predicted by the dynamical theory of Horner. Nonetheless, we show these two results may be reconciled by explicitly verifying that the learning problem for the binary perceptron is NP-complete. This investigation has been made possible by the development of an accelerated annealing algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Minsky, M., Pappert, S.: Perceptrons: An introduction to computational geometry. Cambridge: M.I.T. Press 1969

    Google Scholar 

  2. Hertz, J., Krogh, A., Palmer, R.G.: Introduction to the theory of neutral computation. Reading, Mass: Addison-Wesley 1991

    Google Scholar 

  3. Krauth, W., Mézard, M.: J. Phys. A20, 1745 (1987)

    Google Scholar 

  4. Anlauf, J.K., Biehl, M.: Europhys. Lett.10, 687 (1989)

    Google Scholar 

  5. Gutfreud, H., Stein, Y.: J. Phys. A233, 2613 (1990)

    Google Scholar 

  6. Press, W.H., Flannery, B.P., Teukolsky, S.A., Vetterling, W.T.: Numerical recipes: the art of scientific computing, C.U.P. 1986

  7. Karmarker, N.: Combinatorica4, 373 (1984)

    Google Scholar 

  8. Pitt, L., Vailant, L.G.: J. ACM35(4), 965 (1988)

    Google Scholar 

  9. Cover, T.M.: IEEE Trans. Comput. EC-14, 326 (1965)

    Google Scholar 

  10. Gardner, E.: J. Phys. A21, 257 (1988)

    Google Scholar 

  11. Mézard, M., Parisi, G., Virasoro, M.A.: Spin glass theory and beyond. Singapore: World Scientific 1987

    Google Scholar 

  12. Gardner, E., Derrida, B.: J. Phys. A21, 271 (1988)

    Google Scholar 

  13. Krauth, W., Mézard, M.: J. Phys. A22, 3057 (1989)

    Google Scholar 

  14. Derrida, B.: Phys. Rev. B24, 2631 (1981)

    Google Scholar 

  15. Gross, D.J., Mézard, M.: Nucl. Phys. B240, 431 (1984)

    Google Scholar 

  16. Krauth, W., Opper, M.: J. Phys. A22 L519 (1989)

    Google Scholar 

  17. Horner, H.: Z. Phys. B86, 291 (1992)

    Google Scholar 

  18. Sherrington, D., Kirkpatrick, S.: Phys. Rev. Lett.35, 1792 (1975)

    Google Scholar 

  19. Fu, Y., Lectures in the sciences of complexity. Stein, D. (ed.), p. 815, Reading, Mass: Addison-Wesley 1989

    Google Scholar 

  20. Köhler, H., Diedreich, S., Kinzel, W., Opper, M.: Z. Phys. B78, 333 (1990)

    Google Scholar 

  21. Amaldi, E., Nicolis, S.: J. Phys.50, 2333 (1989)

    Google Scholar 

  22. Perez, C.J., Carrabina, J., Valderrama, E.: Network3, 165 (1992)

    Google Scholar 

  23. Fontanari, J.F., Köberle, R.: J. Phys.51, 1403 (1990)

    Google Scholar 

  24. Köhler, H.: J. Phys. A23, L1265 (1990)

    Google Scholar 

  25. Kirkpatrick, S., Gelatt, Jr., C.D., Vecchi, M.P.: Science220, 671 (1983)

    Google Scholar 

  26. Harland, J.R., Salamon, P.: Nucl. Phys. B (Proc. Suppl)5A, 109 (1988)

    Google Scholar 

  27. Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller: J. Chem. Phys.21, 1087 (1953)

    Google Scholar 

  28. Nulton, J.D., Salamon, P.: Phys. Rev. A37, 1351 (1988)

    Google Scholar 

  29. Horner, H.: (Private communication)

  30. Derrida, B., Griffiths, R.B., Prügel-Bennet, A.: J. Phys. A24, 4907 (1991)

    Google Scholar 

  31. Gardner, E., Derrida, B.: J. Phys. A22, 1983 (1989)

    Google Scholar 

  32. Györghi, G.: J. Phys. Rev. A41, 7097 (1990)

    Google Scholar 

  33. Sompolinsky, H., Tishby, N., Seung, H.S.: Phys. Rev. Lett.65, 1683 (1991)

    Google Scholar 

  34. Horner, H.: Z. Phys. B87, 371 (1992)

    Google Scholar 

  35. Kühn, R.: (Private communication)

  36. Tausworth, R.C.: Math. Commut.19, 201 (1965)

    Google Scholar 

  37. Ahrens, J.H., Dieter, U., Grube, A.: Computing6, 121 (1970)

    Google Scholar 

  38. Dueck, G., Scheuer, T.: J. Comput. Phys.90, 161 (1990)

    Google Scholar 

  39. Creutz, M.: Phys. Rev. Lett.50, 1411 (1983)

    Google Scholar 

  40. Guo, H., Zuckermann, M., Harris, R., Grant, M.: Phys. Scri. T38, 40 (1991)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institut für Theoretische Physik, Ruprecht-Karls-Universität Heidelberg, W-6900, Heidelberg, Germany

    H. -K. Patel

Authors
  1. H. -K. Patel
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Patel, H.K. Computational complexity, learning rules and storage capacities: A Monte Carlo study for the binary perceptron. Z. Physik B - Condensed Matter 91, 257–266 (1993). https://doi.org/10.1007/BF01315244

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01315244

Keywords

Search

Navigation