Selective Ensemble Algorithms of Support Vector Machines Based on Constraint Projection
This paper proposes two novel ensemble algorithms for training support vector machines based on constraint projection technique and selective ensemble strategy. Firstly, projective matrices are determined upon randomly selected must-link and cannot-link constraint sets, with which original training samples are transformed into different representation spaces to train a group of base classifiers. Then, two selective ensemble techniques are used to learn the best weighting vector for combining them, namely genetic optimization and minimizing deviation errors respectively. Experiments on UCI datasets show that both proposed algorithms improve the generalization performance of support vector machines significantly, which are much better than classical ensemble algorithms, such as Bagging, Boosting, feature Bagging and LoBag.
KeywordsSupport vector machines Constraint projection Selective ensemble
Unable to display preview. Download preview PDF.
- 1.Dietterich, T.G.: Machine learning research: four current directions. AI Magazine 18, 97–136 (1997)Google Scholar
- 2.Krogh, A., Vedelsby, J.: Neural Network Ensembles, Cross Validation, and Active Learning. In: Advances in Neural Information Processing Systems, pp. 231–238 (1995)Google Scholar
- 5.Dong, Y.S., Han, K.S.: A Comparison of Several Ensemble Methods for Text Categorization. In: IEEE Int. Conf. on Services Computing, pp. 419–422. IEEE Press, Shanghai (2004)Google Scholar
- 7.Valentini, G., Dietterich, T.: Bias-variance Analysis of Support Vector Machines for the Development of SVM-based Ensemble Methods. J. of Mach. Learn. Res., 725–775 (2004)Google Scholar
- 8.Basu, S., Banerjee, A., Mooney, R.J.: Active Semi-supervision for Pairwise Constrained Clustering. In: Proc. of the SIAM Int. Conf. on Data Mining, Lake Buena Vista, Florida, USA, pp. 333–344 (2004)Google Scholar