Constructive Function Approximation: Theory and Practice
In this paper we study the theoretical limits of finite constructive convex approximations of a given function in a Hilbert space using elements taken from a reduced subset. We also investigate the trade-off between the global error and the partial error during the iterations of the solution. These results are then specialized to constructive function approximation using sigmoidal neural networks. The emphasis then shifts to the implementation issues associated with the problem of achieving given approximation errors when using a finite number of nodes and a finite data set for training.
Unable to display preview. Download preview PDF.
- P. Auer, M. Herbster, and M.K. Warmuth. Exponentially many local minima for single neurons. In D. Touretzky, M.C. Mozer, and M.E. Hasselmo, editors, Advances in Neural Information Processing Systems 8, pages 316–322. Morgan Kaufmann, 1996.Google Scholar
- A.R. Barron. Statistical properties of artificial neural networks. In Proceedings of the 28th IEEE Conf. on Decision and Control, pages 280–285, 1989.Google Scholar
- A.R. Barron. Approximation and estimation bounds for artificial neural networks. In L.G. Valiant and M.K. Warmuth, editors, Proceedings of the 4th Annual Workshop on Computational Learning Theory, pages 243–249, 1991.Google Scholar
- R. Battiti. First-and second—order methods for learning: between steepest descent and newton’s method. Neural Computation, 4(2):141–166, 1992.Google Scholar
- L. Breiman and J.H. Friedman. Function approximation using ramps. In Snowbird Workshop on Machines that Learn, 1994.Google Scholar
- L. Breiman. Hinging hyperplanes for regression, classification and function approximation. IEEE Trans. on Inf. Theory, 39(3), 1993.Google Scholar
- E.W. Cheney. Topics in approximation theory, 1992.Google Scholar
- J.H. Friedman. Multivariate adaptive regression splines. The Annals of Statistics, 19, 1991.Google Scholar
- F. Girosi and G. Anzellotti. Convergence rates of approximation by translates. Technical Report 1288, MIT Art. Intell. Lab., 1992.Google Scholar
- Simon Haykin. Neural Networks: A Comprehensive Foundation. Macmillan, New York, 1992.Google Scholar
- G. Pisier. Remarques sur un resultat non publié de b. maurey, 1980–1981.Google Scholar
- D.E. Rumelhart, G.E. Hinton, and R.J. Williams. Learning internal representations by error propagation. In D.E. Rumelhart and J.L. McClelland, editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, pages 318–362. MIT Press, Cambridge, MA, 1986.Google Scholar
- Y. Zhao. On projection pursuit learning. PhD thesis, Dept. Math. Art. Intell. Lab., MIT, Boston, MA, 1992.Google Scholar