Improving the performance of Piecewise linear Separation incremental algorithms for practical hardware implementations
In this paper we shall review the common problems associated with Piecewise Linear Separation incremental algorithms. This kind of neural models yield poor performances when dealing with some classification problems, due to the evolving schemes used to construct the resulting networks. So as to avoid this undesirable behavior we shall propose a modification criterion. It is based upon the definition of a function which will provide information about the quality of the network growth process during the learning phase. This function is evaluated periodically as the network structure evolves, and will permit, as we shall show through exhaustive benchmarks, to considerably improve the performance (measured in terms of network complexity and generalization capabilities) offered by the networks generated by these incremental models.
Unable to display preview. Download preview PDF.
- J.M. Moreno, “VLSI Architectures for Evolutive Neural Models”, Ph. D. thesis. Universitat Politécnica de Catalunya, 1994.Google Scholar
- Y. Kwok, D.-Y. Yeung, “Constructive Feedforward Neural Networks for Regression Problems: A Survey”, Technical Report HKUST-CS95-43, Hong Kong University of Science and Technology, 1995.Google Scholar
- J.A., Sirat, J.P. Nadal, “Neural Trees: A New Tool for Classification”, Technical Report, Laboratoires d'Electronique Philips, 1990.Google Scholar
- M. Rosenblatt, “Principles of Neurodynamics”, Spartan, New York, 1962.Google Scholar
- S.I. Gallant, “Optimal Linear Discriminants”. Proc. of the 8th Intl. Conf. on Pattern Recognition, pps. 849–854, Paris, 1988.Google Scholar
- J.M. Moreno, F. Castillo, J. Cabestany, “Optimized Learning for Improving the Evolution of Piecewise Linear Separation Incremental Algorithms”, New Trends in Neural Computation, J. Mira, J. Cabestany, A. Prieto (eds.), pps. 272–277, Springer-Verlag, 1993.Google Scholar
- J.M. Moreno, F. Castillo, J. Cabestany, “Improving Piecewise Linear Separation Incremental Algorithms Using Complexity Reduction Methods”, Proc. of the European Symposium on Artificial Neural Networks, ESANN'94, pps. 141–146, 1994.Google Scholar
- E.B. Baum, D. Hausler, “What Size Net Gives Valid Generalization”, Neural Computation, Vol. 1, pps. 151–160, 1989.Google Scholar
- B. Efron, “Bootstrap Methods: Another Look at the Jacknife”, The Annals of Statistics, Vol. 7, No. 1, pps. 1–26, 1979.Google Scholar
- Murata, S. Yoshizawa, S.-I. Amari, “Network Information Criterion. Determining the Number of Hidden Units for an Artificial Neural Network Model”, IEEE Trans. on Neural Networks, Vol. 5, No 6, pps. 865–872, November 1994.Google Scholar