Abstract
The statistical physics of disordered systems provides tools for the investigation of learning processes in adaptive information processing. The methods and objectives of this approach are exemplified in terms of a specific model scenario: the supervised learning of a rule with a multilayered neural network. The model exhibits a discontinuous dependence of the student performance on the number of example data. This phenomenon can be interpreted as a symmetry breaking phase transition, which results from the competition of (formal) energy and entropy.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Bibliography
J.A. Hertz, A. Krogh R.G. Plamer, Introduction to the Theory of Neural Computation. (Addison-Wesley, Redwood City (CA) 1991).
C. Bishop, Neural Networks for Pattern Recognition. (Clarendon, Oxford, 1995).
M. Opper and W. Kinzel, in Models of Neural Networks III eds. E. Domany, J.L. van Hemmen, and K. Schulten, (Springer, Berlin 1996).
S. Seung, H. Sompolinsky, and N. Tishby, Phys. Rev. A 45, 6056 (1992).
T.L.H. Watkin, A. Rau, and M. Biehl, Rev. Mod. Phys. 65, 499 (1993).
W. Kinzel, Phil. Mag. B 77, 1455 (1998).
Y. Chauvin and D.E. Rumelhart (eds.), Backpropagation: Theory, Architecture, and Applications. (Lawrence Erlbaum, Hillsdale (NJ) 1995).
M. Biehl and H. Schwarze, J. Phys. A 28, 643 (1995).
D. Saad and S.A. Solla, Phys. Rev. E 52, 4225 (1995).
H. Schwarze and J. Hertz, Europhys. Lett. 21, 785 (1993).
H. Schwarze, J. Phys. A 26, 5781 (1993).
K. Kang, J.-H. Oh, C. Kwon, and Y. Park, Phys. Rev. E 48, 4805 (1993).
R. Urbanczik, J. Phys A 28, 7097, (1995) and Phys. Rev. E 58, 2298 (1998).
B. Schottky, J. Phys A 28, 4515 (1995).
B. Schottky and U. Krey, J. Phys A 30, 8541 (1997).
H. Schwarze, M. Opper, and W. Kinzel, Phys. Rev. A 46, R6185 (1992).
M. Ahr, M. Biehl, and R. Urbanczik, Eur. Phys. J. B 10, 583 (1999).
M. Biehl, E. Schlösser, and M. Ahr, Europhys. Lett. 44, 261 (1998).
M. Ahr, M. Biehl, and R. Urbanczik, J. Phys. A. 32, L531 (1999).
R. Dietrich, M. Opper, and H. Sompolinsky, Phys. Rev. Lett. 82, 2975 (1999).
C. Marangi, M. Biehl, and S.A. Solla, Europhys. Lett. 30, 117 (1995).
M. Biehl and A. Mietzner, Europhys. Lett. 24, 421 (1993).
P. Reimann, C. Van den Broeck, and G.J. Bex, J. Phys. A 29, 3521 (1996).
M. Biehl, P. Riegler, and C. Wöhler, J. Phys. A 29, 4769 (1996).
D. Saad (ed.), On-line learning in neural networks. (Cambridge University Press, 1998).
M. Biehl and N. Caticha, in The Handbook of Brain Theory and Neural Networks, ed. M.A. Arbib, MIT Press (in press).
O. Kinouchi and N. Caticha, J. Phys. A 28, 6243 (1992).
R. Vicente and N. Caticha, J. Phys. A 30, L559 (1997).
D. Saad and M. Rattray, Phys. Rev. Lett. 79, 2578 (1997).
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2000 Friedr. Vieweg & Sohn Verlagsgesellschaft mbH
About this paper
Cite this paper
Biehl, M., Ahr, M., Schlösser, E. (2000). Statistical physics of learning: Phase transitions in multilayered neural networks. In: Kramer, B. (eds) Advances in Solid State Physics 40. Advances in Solid State Physics, vol 40. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0108398
Download citation
DOI: https://doi.org/10.1007/BFb0108398
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-41576-3
Online ISBN: 978-3-540-44560-9
eBook Packages: Springer Book Archive