Polynomial Network Classifier with Discriminative Feature Extraction
The polynomial neural network, or called polynomial network classifier (PNC), is a powerful nonlinear classifier that can separate classes of complicated distributions. A method that expands polynomial terms on principal subspace has yielded superior performance. In this paper, we aim to further improve the performance of the subspace-feature-based PNC. In the framework of discriminative feature extraction (DFE), we adjust the subspace parameters together with the network weights in supervised learning. Under the objective of minimum squared error, the parameters can be efficiently updated by stochastic gradient descent. In experiments on 13 datasets from the UCI Machine Learning Repository, we show that DFE can either improve the classification accuracy or reduce the network complexity. On seven datasets, the accuracy of PNC is competitive with support vector classifiers.
KeywordsLinear Discriminant Analysis Network Weight Stochastic Gradient Descent Polynomial Term Fisher Linear Discriminant Analysis
- 2.Shürmann, J.: Pattern Classification: A Unified View of Statistical and Neural Approaches. Wiley Interscience, Chichester (1996)Google Scholar
- 5.Kreßel, U., Schürmann, J.: Pattern classification techniques based on function approximation. In: Bunke, H., Wang, P.S.P. (eds.) Handbook of Character Recognition and Document Image Analysis, pp. 49–78. World Scientific, Singapore (1997)Google Scholar
- 7.Shin, Y., Ghosh, J.: The Pi-sigma network: an efficient higher-order neural network for pattern classification and function approximation. In: Proc. IJCNN 1991, Seattle, vol. 1, pp. 13–18 (1991)Google Scholar
- 16.UCI Machine Learning Repository, http://www.ics.uci.edu/~mlearn/MLRepository.html