Abstract
Techniques from statistical physics have been applied successfully in recent years to the analysis of the generalization performance of neural networks. However, most of the analysis to date has been for perceptron-like networks or simple generalizations thereof such as committee machines, and none of the networks studied are used to any significant extent in practice. This letter presents results obtained in applying techniques from statistical physics to a popular class of neural networks that has been used successfully in many practical applications: the Gaussian radial basis function networks. We obtain expressions for the learning curves exhibited by these networks in the high-temperature limit for both realizable and unrealizable target rules.
Similar content being viewed by others
References
H. S. Seung, H. Sompolinsky, N. Tishby. Statistical mechanics of learning from examples,Physical Review A, vol. 45, no. 8, pp. 6056–6091, April 1992.
T. L. H. Watkin, A. Rau, M. Biehl. The statistical mechanics of learning a rule,Rev. Mod. Phys., vol. 65, no. 2, pp. 499–556, April 1993.
H. Schwarze, M. Opper, W. Kinzel. Generalization in a two-layer neural network,Physical Review A, vol. 46, no. 10, pp. R6185-R6188, 1992.
E. Barkai, D. Hansel, H. Sompolinsky. Broken symmetries in multilayered perceptrons,Physical Review A, vol. 45, no. 6, pp. 4146–4161, March 1992.
D. S. Broomhead, David Lowe. Multivariable functional interpolation and adaptive networks,Complex Systems, vol. 2, pp. 321–355, 1988.
M. J. D. Powell. The theory of radial basis function approximation in 1990,Technical report #DAMTP 1990/NA11, Department of Applied Mathematics and Theoretical Physics, University of Cambridge, December 1990.
T. Poggio, F. Girosi. Networks for approximation and learning,Proceedings of the IEEE, vol. 78, no. 9, pp. 1481–1497, September 1990.
S. Renals, R. Rohwer. Phoneme classification experiments using radial basis functions,Proceedings of the International Joint Conference on Neural Networks, pp. I-461–I-467, June 1989.
M. Niranjan, F. Fallside. Neural networks and radial basis functions in classifying static speech patterns,Computer Speech and Language, vol. 4, no. 3, pp. 275–289, July 1990.
S. B. Holden, P. J. W. Rayner, Generalization and PAC learning: Some new results for the class of generalized single layer networks,IEEE Transactions on Neural Networks, vol. 6, no. 2, pp. 368–380, March 1995.
M. Anthony, S. B. Holden. Quantifying generalization in linearly weighted neural networks,Complex Systems, vol. 8, pp. 91–114, 1994.
E. Levin, N. Tishby, S. A. Solla. A statistical approach to learning and generalization in layered neural networks,Proceedings of the IEEE, vol. 78, no. 10, pp. 1568–1574, October 1990.
S. B. Holden, M. Niranjan. Average-case learning curves for radial basis function networks,Technical Report CUED/F-INFENG/TR.212, Cambridge University Engineering Department, Trumpington Street, Cambridge CB2 1PZ, 1995.
S. Godsill.The restoration of degraded audio signals, Ph.D. thesis, Cambridge University Engineering Department, Trumpington Street, Cambridge CB2 1PZ, U.K., 1993.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Holden, S.B., Niranjan, M. On the statistical physics of radial basis function networks. Neural Process Lett 2, 16–19 (1995). https://doi.org/10.1007/BF02279933
Issue Date:
DOI: https://doi.org/10.1007/BF02279933