Optimality of pocket algorithm

  • Marco Muselli
Poster Presentations 1 Theory II: Learning
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1112)


Many constructive methods use the pocket algorithm as a basic component in the training of multilayer perceptrons. This is mainly due to the good properties of the pocket algorithm confirmed by a proper convergence theorem which asserts its optimality.

Unfortunately the original proof holds vacuously and does not ensure the asymptotical achievement of an optimal weight vector in a general situation. This inadequacy can be overcome by a different approach that leads to the desired result.

Moreover, a modified version of this learning method, called pocket algorithm with ratchet, is shown to obtain an optimal configuration within a finite number of iterations independently of the given training set.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hertz, J., Krogh, A., and Palmer, R. G.Introduction to the Theory of Neural Computation. Redwood City, CA: Addison-Wesley, 1991.Google Scholar
  2. 2.
    Mézard, M., and Nadal, J.-P. Learning in feedforward layered networks: The tiling algorithm. Journal of Physics A22 (1989), 2191–2203.Google Scholar
  3. 3.
    Frean, M. The upstart algorithm: A method for constructing and training feed-forward neural networks. Neural Computation2 (1990), 198–209.Google Scholar
  4. 4.
    Muselli, M. On sequential construction of binary neural networks. IEEE Transactions on Neural Networks6 (1995), 678–690.Google Scholar
  5. 5.
    Gallant, S. I. Perceptron-based learning algorithms. IEEE Transactions on Neural Networks1 (1990), 179–191.Google Scholar
  6. 6.
    Rosenblatt, F.Principles of Neurodynamics. Washington, DC:Spartan Press, 1961.Google Scholar
  7. 7.
    Muselli, M. On convergence properties of pocket algorithm. Submitted for publication on IEEE Transactions on Neural Networks.Google Scholar
  8. 8.
    Minsky, M., and Papert, S.Perceptrons: An Introduction to Computational Geometry. Cambridge, MA: MIT Press, 1969.Google Scholar
  9. 9.
    Nummelin, E.General Irreducible Markov Chains and Non-Negative Operators. New York: Cambridge University Press, 1984.Google Scholar
  10. 10.
    Godbole, A. P. Specific formulae for some success run distributions. Statistics & Probability Letters10 (1990), 119–124.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1996

Authors and Affiliations

  • Marco Muselli
    • 1
  1. 1.Istituto per i Circuiti ElettroniciConsiglio Nazionale delle RicercheGenovaItaly

Personalised recommendations