Radial Basis Function Networks 2 pp 1-36 | Cite as

# An Overview of Radial Basis Function Networks

## Abstract

This chapter presents a broad overview of Radial Basis Function Networks (RBFNs), and facilitates an understanding of their properties by using concepts from approximation theory, catastrophy theory and statistical pattern recognition. While this chapter is aimed to provide an adequate theoretical background for the subsequent application oriented chapters in this book, it also covers several aspects with immediate practical implications: alternative ways of training RBFNs, how to obtain an appropriate network size for a given problem, and the impact of the resolution (width) of the radial basis functions on the solution obtained. Some prominent applications of RBFNs are also outlined.

## Keywords

Basis Function Radial Basis Function Radial Basis Function Network Hide Unit Ridge Regression## Preview

Unable to display preview. Download preview PDF.

## References

- [1]Ahmad, S. and Tresp, V. (1993), “Some solutions to missing feature problem in vision,” in Cowan, J.D., Hanson, S.J., and Giles, C.L. (Eds.),
*Advances in Neural Information Processing Systems*, vol. 5, pp. 393–400. The MIT Press.Google Scholar - [2]Bianchini, M., Frasconi, P., and Gori, M. (1995), “Learning without local minima in radial basis function networks,”
*IEEE Transactions on Neural Networks*, vol. 6, no. 3, pp. 749–756.CrossRefGoogle Scholar - [3]Beck, S. and Ghosh, J. (1992), “Noise sensitivity of static neural classifiers,”
*SPIE Conf. on Applications of Artificial Neural Networks*,*SPIE Proc. Vol*.*1709*, pp. 770–779, Orlando, Fl., April.Google Scholar - [4]Bishop, C.M. (1995),
*Neural Networks for Pattern Recognition*, Oxford University Press, New York.Google Scholar - [5]Broomhead, D.S. and Lowe, D. (1988), “Multivariable functional interpolation and adaptive networks,”
*Complex Systems*, vol. 2, pp. 321–355.MathSciNetMATHGoogle Scholar - [6]Bor§, A.G. and Pitas, I. (1996), “Median radial basis function neural network,”
*IEEE Transactions on Neural Networks*, vol. 7, no. 6, pp. 1351–1364, November.Google Scholar - [7]Bakshi, B.R. and Stephanopoulos, G. (1993), “Wave-net: a multiresolution, hierarchical neural network with localized learning,”
*AIChE Journal*, vol. 39, no. 1, pp. 57–81, January.Google Scholar - [8]Le Cun, Y., Denker, J.S., and Solla, S.A. (1990), “Optimal brain damage,”
*Advances in Neural Information Processing Systems-2*, pp. 598–605.Google Scholar - [9]Chakaravathy, S.V. and Ghosh, J. (1996), “Scale based clustering using a radial basis function network,”
*IEEE Transactions on Neural Networks*,*vol*. 5, no. 2, pp. 1250–61, Sept.Google Scholar - [10]Chakaravathy, S.V. and Ghosh, J. (1997), “Function emulation using radial basis function networks,”
*Neural Networks*, vol. 10, pp. 459–478, May.Google Scholar - [11]Chakravarthy, S.V. (1996),
*On the role of singularities in neural networks*, PhD thesis, Dept. of Elect. Comp. Eng., Univ of Texas, Austin, May.Google Scholar - [12]Friedman, J.H. (1994), “An overview of predictive learning and function approximation,” in Cherkassky, V., Friedman, J.H., and Wechsler, H. (Eds.),
*From Statistics to Neural Networks*,*Proc. NATO/ASI Workshop*, pp. 1–61. Springer Verlag.Google Scholar - [13]Geman, S., Bienenstock, E., and Doursat, R. (1992), “Neural networks and the bias/variance dilemma,”
*Neural Computation*, vol. 4, no. 1, pp. 1–58.CrossRefGoogle Scholar - [14]Ghosh, J. and Chakravarthy, S.V. (1994), “The rapid kernel classifier: a link between the self-organizing feature map and the radial basis function network,”
*Jl. of Intelligent Material Systems and Structures*, vol. 5, pp. 211–219, 2.Google Scholar - [15]Ghosh, J., Deuser, L., and Beck, S. (1992), “A neural network based hybrid system for detection, characterization and classification of short-duration oceanic signals,”
*IEEE X. of Ocean Engineering*, vol. 17, no. 4, pp. 351–363, October.Google Scholar - [16]Gilmore, R. (1981),
*Catastrophe Theory for Scientists and Engineers*, Wiley Interscience, New York.MATHGoogle Scholar - [17]Ghahramani, Z. and Jordan, M. (1994), “Supervised learning from incomplete data via an em approach,” in Tesauro, G., Cowan, J.D., and Alspector, J. (Eds.),
*Advances in Neural Information Processing Systems, vol*. 6, pp. 120–127. The MIT Press.Google Scholar - [18]Golub, G. and Van Loan, C. (1989),
*Matrix Computations*, John Hopkins University Press, Baltimore, MD.Google Scholar - [19]Ghosh, J. and Tumer, K. (1994), “Structural adaptation and generalization in supervised feedforward networks,”
*fi. of Artificial Neural Networks*, vol. 1, no. 4, pp. 431–458.Google Scholar - [20]Haykin, S. (1994),
*Neural Networks: a Comprehensive Foundation*, Macmillan, New York.MATHGoogle Scholar - [21]Hoskins, J.C., Lee, P., and Chakravarthy, S.V. (1993), “Polynomial modelling behavior in radial basis function networks,”
*Proc. of World Congress on Neural Networks*, pp. 693–699, Portland, OR, July.Google Scholar - [22]Jong, J.S.R. and Sun, C.T. (1993), “Functional equivalence between radial basis function networks and fuzzy inference systems,”
*IEEE Transactions on Neural Networks*, vol. 4, no. 1, pp. 156–159, January.Google Scholar - [23]Krogh, A. and Hertz, J.A. (1992), “A simple weight decay can improve generalization,” in Hanson, S.J., Moody, J.E., and Lippmann, R.P. (Eds.),
*Advances in Neural Information Processing Systems-4*, pp. 950–957. Morgan Kaufmann, San Mateo, CA.Google Scholar - [24]Kowalski, J., Hartman, E., and Keeler, J. (1990), “Layered neural networks with gaussian hidden units as universal approximators,”
*Neural Computation*, vol. 2, pp. 210–215.CrossRefGoogle Scholar - [25]Krzyzak, A. and Linder, T. (1997), “Radial basis function networks and complexity regularization in function learning,” in Mozer, M.C., Jordan, M.I., and Petsche, T. (Eds.),
*Advances in Neural Information Processing Systems, vol*. 9, p. 197.Google Scholar - [26]Krzyzak, A., Linder, T., and Lugosi, G. (1996), “Nonparametric estimation and classification using radial basis function nets and empirical risk minimization,”
*IEEE Transactions on Neural Networks*, vol. 7, no. 2, pp. 475–487, March.Google Scholar - [27]Kadirkamanathan, V. and Niranjan, M. (1993), “A function estimation approach to sequential learning with neural networks,”
*Neural Computation, vol*. 5, pp. 954–975.CrossRefGoogle Scholar - [28]Light, W.A. (1992), “Some aspects of radial basis function approximation,” in Singh, S.P. (Ed.),
*Approximation Theory, Spline Functions and Applications*, pp. 163–90. NATO ASI Series Vol. 256, Kluwer Acad., Boston.Google Scholar - [29]Lippmann, R.P. (1991), “A critical overview of neural network pattern classifiers,”
*IEEE Workshop on Neural Networks for Signal Processing*.Google Scholar - [30]Lee, S. and Kil, R.M. (1988), “Multilayer feedforward potential function network,”
*Proceedings of the Second International Conference on Neural Networks*, pp. 161–171.Google Scholar - [31]Lee, S. and Kil, R.M. (1991), “A Gaussian potential function network with hierarchical self-organizing learning,”
*Neural Networks*, vol. 4, pp. 207–224.CrossRefGoogle Scholar - [32]Leonard, J.A., Kramer, M.A., and Ungar, L.H. (1992), “Using radial basis functions to approximate a function and its error bounds,”
*IEEE Transactions on Neural Networks*, vol. 3, no. 4, pp. 624–627, July.Google Scholar - [33]Mackay, D.J.C. (1995), “Probable networks and plausible predictions–a review of practical Bayesian methods for supervised neural networks,”
*Network: Computation in Neural Systems*, vol. 6, no. 3, pp. 469–505.MATHCrossRefGoogle Scholar - [34]Moody, J. and Darken, C.J. (1989), “Fast learning in networks of locally-tuned processing units,”
*Neural Computation*, vol. 1, no. 2, pp. 281–294.CrossRefGoogle Scholar - [35]Megdassy, P. (1961),
*Decomposition of superposition of distributed functions*, Hungarian Academy of Sciences, Budapest.Google Scholar - [36]Micchelli, C.A. (1986), “Interpolation of scattered data: distance matrices and conditionally positive definite functions,”
*Constructive Approximation*, vol. 2, pp. 11–22.MathSciNetMATHCrossRefGoogle Scholar - [37]Molina, C. and Niranjan, M. (1996), “Pruning with replacement on limited resource allocating networks by F-projections,”
*Neural Computation*, vol. 8, pp. 855–868.CrossRefGoogle Scholar - [38]Moody, J.E. (1994), “Prediction risk and architecture selection for neural networks,” in Cherkassky, V., Friedman, J.H., and Wechsler, H. (Eds.),
*From Statistics to Neural Networks*,*Proc. NATO/ASI Workshop*, pp. 143–156. Springer Verlag.Google Scholar - [39]Mulgrew, B. (1996), “Applying radial basis functions,”
*IEEE Signal Processing Magazine*, pp. 50–65, March.Google Scholar - [40]Nadaraya, E.A. (1964), “On estimating regression,”
*Theory of Probability and its Applications*, vol. 9, no. 1, pp. 141–142,.Google Scholar - [41]Niyogi, P. and Girosi, F. (1996), “On the relationship between generalization error, hypothesis complexity and sample complexity for radial basis functions,”
*Neural Computation*, vol. 8, pp. 819–842.CrossRefGoogle Scholar - [42]Osman, H. and Fahmy, M.M. (1994), “Probabilistic winner-takeall learning algorithm for radial-basis-function neural classifiers,”
*Neural Computation*, vol. 6, no. 5, pp. 927–943.MATHCrossRefGoogle Scholar - [43]Orr, M.J.L. (1995), “Regularization in the selection of radial basis function centers,”
*Neural Computation*, vol. 7, pp. 606–623.CrossRefGoogle Scholar - [44]Orr, M.J.L. (1996), “Introduction to radial basis function networks,” Technical Report April, Center for Cognitive Science, Univ. of Edinburgh.Google Scholar
- [45]Poggio, T. and Girosi, F. (1990), “Networks for approximation and learning,”
*Proc. IEEE*, vol. 78, no. 9, pp. 1481–97, Sept.Google Scholar - [46]Platt, J.C. (1991), “A resource allocation network for function interpolation,”
*Neural Computation*, vol. 3, no. 2, pp. 213–225.MathSciNetCrossRefGoogle Scholar - [47]Pal, S.K. and Mitra, S. (1992), “Multilayer perceptron, fuzzy sets, and classification,”
*IEEE Transactions on Neural Networks*, vol. 3, no. 5, pp. 683–697, September.Google Scholar - [48]Powell, M.J.D. (1985), “Radial basis functions for multivariable interpolation: a review,”
*IMA Conf on Algorithms for the approximation of functions and data*, pp. 143–167.Google Scholar - [49]Park, J. and Sandberg, I.W. (1991), “Universal approximation using radial basis function networks,”
*Neural Computation*, vol. 3, no. 2, pp. 246–257, Summer.Google Scholar - [50]Park, J. and Sandberg, I.W. (1993), “Universal approximation and radial basis function networks,”
*Neural Computation*, vol. 5, no. ??, pp. 305–316.Google Scholar - [51]Reed, R. (1993), “Pruning algorithms — a survey,”
*IEEE Transactions on Neural Networks*, vol. 4, no. 5, pp. 740–747, September.Google Scholar - [52]Rosenblum, M., Yacoob, Y., and Davis, L.S. (1996), “Human expression recognition from motion using a radial basis function network architecture,”
*IEEE Transactions on Neural Networks*, vol. 7, no. 5, pp. 1121–1138, September.Google Scholar - [53]Shim, C. and Cheung, J.Y. (1995), “Pattern classification using RBF based fuzzy neural network,”
*Intelligent Engineering Systems Through Artificial Neural Networks*, vol. 5, pp. 485–90. ASME Press, November.Google Scholar - [54]Schioler, H. and Hartmann, U. (1992), “Mapping neural network derived from the parzen window estimator,”
*Neural Networks*, vol. 5, pp. 903–909.CrossRefGoogle Scholar - [55]Specht, J. (1990), “Probabilistic neural networks,”
*Neural Networks*, vol. 3, pp. 45–74.CrossRefGoogle Scholar - [56]Tresp, R., Neuneier, V., and Ahmad, S. (1995), “Efficient methods for dealing with missing data in supervised learning,” in Touretzky, D.S., Tesauro, G., and Leen, T.K. (Eds.),
*Advances in Neural Information Processing Systems*, vol. 7, pp. 687–696. The MIT Press.Google Scholar - [57]Taha, I. and Ghosh, J. (1997), “Hybrid intelligent architecture and its application to water reservoir control,”
*International Journal of Smart Engineering Systems*, vol. 1, pp. 59–75, 1.Google Scholar - [58]Thom, R. (1975),
*Structural Stability and Morphogenesis*, Reading: Benjamin.MATHGoogle Scholar - [59]Tresp, S., Ahmad, V., and Neuneier, R. (1994), “Training neural networks with deficient data,” in Tesauro, G., Cowan, J.D., and Alspector, J. (Eds.),
*Advances in Neural Information Processing Systems*, vol. 6, pp. 128–135. The MIT Press.Google Scholar - [60]Wahba, G. (1990),
*Spline Models for Observational Data*, vol. 59. Society for Industrial and Applied Mathematics, Philadephia.MATHCrossRefGoogle Scholar - [61]Watson, G.S. (1964), “Smooth regression analysis,”
*Sankhya: the Indian Journal of Statistics*, Series A 26, pp. 359–372.Google Scholar - [62]Webb, A.R. (1996), “An approach to non-linear principal component analysis using radially symmetric kernel functions,”
*Statistics and Computing, vol*. 9, pp. 159–68.CrossRefGoogle Scholar - [63]Witkin, A.P. (1983), “Scale-space filtering,”
*8th Int. Joint Conf. Art. Intell.*, pp. 1019–1022, Karlsruhe, Germany, August.Google Scholar - [64]Wong, Y.F. (1993), “Clustering data by melting,”
*Neural Computation, vol*. 5, no. 1, pp. 89–104.CrossRefGoogle Scholar - [65]Webb, A.R. and Shannon, S.M. (1996), “Spherically-symmetric basis functions for discrimination and regression: determining the nonlinearity,” Technical Report 651/2/JP95/22, DRA/CIS, Malvern, England.Google Scholar
- [66]Xu, L., Krzyzak, A., and Yuille, A. (1994), “On radial basis function nets and kernel regression: statistical consistency, convergence rates and receptive field size,”
*Neural Networks*, vol. 7, no. 4, pp. 609–628.MATHCrossRefGoogle Scholar - [67]Yingwei, L., Sunderararajan, N., and Saratchandran, P. (1997), “A sequential learning scheme for function approximation using minimal radial basis function networks,”
*Neural Computation*, vol. 9, pp. 461–78.MATHCrossRefGoogle Scholar