A Novel Selective Ensemble Learning Based on K-means and Negative Correlation
Selective ensemble learning has drawn high attention for improving the diversity of the ensemble learning. However, the performance is limited by the conflicts and redundancies among its child classifiers. In order to solve these problems, we put forward a novel method called KNIA. The method mainly makes use of K-means algorithm, which is used in the integration algorithm as an effective measure to choose the representative classifiers. Then, negative correlation theory is used to select the diversity of classifiers derived from the representative classifiers. Compared with the classical selective learning, our algorithm which is inverse growth process can improve the generalization ability in the condition of ensuring the accuracy. The extensive experiments demonstrate that the robustness and precision of the proposed method outperforms four classical algorithms from multiple UCI data sets.
KeywordsEnsemble learning K-means Negative correlation Neural network
The author is grateful to Baosheng Wang and Bo Yu for the guidance and advice, and thanks to the support of the project. The work was supported by Science Foundation of China under (NSFC) Grant No. 61472437 and No. 61303264.
- 2.Gu, B., Sheng, V.S.: A robust regularization path algorithm for -support vector classification. IEEE Trans. Neural Netw. Learn. Syst. (2016). doi: 10.1109/TNNLS.2016.2527796
- 3.Gu, B., Sun, X., Sheng, V.S.: Structural minimax probability machine. IEEE Trans. Neural Netw. Learn. Syst. (2016). doi: 10.1109/TNNLS.2016.2544779
- 5.Dietterich, T.G.: Machine learning research: four current directions. AI Mag. 18(4), 97–136 (1977)Google Scholar
- 9.Krogh, P.S.A.: Learning with ensembles: how over-fitting can be useful. In: Proceedings of the 1995 Conference, vol. 8, p. 190 (1996)Google Scholar
- 14.Cheng, X., Guo, H.: The technology of selective multiple classifiers ensemble based on kernel clustering. In: Second International Symposium on Intelligent Information Technology Application, IITA 2008, vol. 2, pp. 146–150. IEEE (2008)Google Scholar
- 23.Fu, Q., Hu, S.X., Zhao, S.Y.: A PSO-based approach for neural network ensemble. J. Zhejiang Univ. (Eng. Sci.) 38(12), 1596–1600 (2004)Google Scholar
- 24.Margineantu, D.D., Dietterich, T.G.: Pruning adaptive boosting. ICML 97, 211–218 (1997)Google Scholar