Double SVMSBagging: A Subsampling Approach to SVM Ensemble
In ensemble methods, pooling the decisions of multiple unstable classifiers often lead to improvements in the generalization performance substantially in many applications. We propose here a new ensemble method, Double SVMSBagging, which is a variant of double bagging. In this method we have used subsampling in order to make the out-of-bag samples larger and trained support vector machine as the additional classifier on these out-of-bag samples. The underlying base classifier is the decision tree. We have used radial basis function kernel, expecting that the new classifier can perform efficiently in both linear and non-linear feature space. We have studied the performance of the proposed ensemble method in several benchmark datasets with different subsampling rate (SSR). We have applied the proposed method in partial discharge classification of the gas insulated switchgear (GIS). We compare the performance of double SVMsbagging with other well-known classifier ensemble methods in condition diagnosis; the double SVMsbagging performed better than other ensemble method in this case. We applied the double SVMsbagging in 15 UCI benchmark datasets and compare its accuracy with other ensemble methods, e.g., Bagging, Adaboost, Random Forest and Rotation Forest. The performance of this method with optimum SSR generate significantly lower prediction error than Rotation Forest and Adaboost for most of the datasets.
KeywordsSupport vector machine Double bagging Subsampling rate CART Partial discharge analysis
- 1.Blake. C., & Merz, C. (1999). UCI repository of machine learning databases, http://www.ics.uci.edu/mlearn/MLRepository.html.
- 6.Breiman, L. (2001). Statistical modeling: the two cultures. Statistical Science, 16(3), 199–231 (with discussion).Google Scholar
- 8.Chang, C., & Lin, C. (2001). LIBSVM: a library for support vector machines, software available at http://www.csie.ntu.edu.tw/cjlin/libsvm.
- 10.Freund Y., & Schapire, R. (1996). Experiments with a new boosting algorithm. Proceedings of the thirteenth international conference machine learning (pp. 148–156). San Francisco, MA: Morgan Kaufmann.Google Scholar
- 11.Friedman, J., Hastie, T., & Tibshirani, R. (2000). Additive logistic regression: a statistical view of boosting. Annals of Statistics, 28, 337–407(with discussion).Google Scholar
- 14.Hirose, H., Matsuda, S., & H., Hikita. (2006). Electrical insulation diagnosing using a new statistical classification method. Proceedings of the 8th international conference on properties and applications of dielectric materials (ICPADM2006) (pp. 698–701).Google Scholar
- 16.Hirose, H., Zaman, F., Tsuru, K., Tsuboi, T., & Okabe, S. (2008). Diagnosis accuracy in electric power apparatuses conditions using the classification methods. IEICE Technical Report, 108(243), 39–44.Google Scholar
- 18.Iba, W., & Langley, R. (1992). Induction of one-level decision trees. Proceedings of the nineteenth international conference on machine learning, Aberdeen, Scotland.Google Scholar
- 19.Joachims, T. (1999). Making large-scale support vector machine learning practical. Advances in kernel methods: support vector machines (pp. 169–184). Cambridge, MA: MIT Press.Google Scholar
- 20.Li, Y., Cal, Y., Yin, R., & Xu, X. (2005). Fault diagnosis based on support vector machine ensemble. Proceedings of the 2005 international conference on machine Learning. Cybernet, 6, 3309–3314.Google Scholar
- 23.Patton, R., Lopez-Toribio, C., & Uppal, F. (1999). Artificial intelligence approaches to fault diagnosis, condition monitoring. IEE Colloquium on Machinery, External Structures and Health (Ref. No. 1999/034), 5/1–5/18.Google Scholar
- 24.Platt, J. (1999). Fast training of support vector machines using sequential minimal optimization. In B. Scholkopf, C.J.C. Burges, & A.J. Smola (Eds.), Advances Kernel methods – support vector learning (pp. 185–208). Cambridge, MA: MIT.Google Scholar
- 25.Platt, J. (2000). Probabilistic outputs for support vector machines and comparison to regularized likelihood methods. In A.J. Smola, P. Bartlett, B. Scholkopf, & D. Schuurmans, (Eds.), Advances in large margin classifiers (pp. 61–74). Cambridge, MA: MIT Press.Google Scholar
- 28.Scholkopf, B., Burges, C., & Smola, A. (1999). Introduction to support vector learning. In B. Scholkopf, C.J.C. Burges, & A.J. Smola (Eds.), Advances in kernel methods: support vector learning (pp. 1–15). Cambridge, MA: MIT Press.Google Scholar
- 29.Sorsa, T. (1995). Neural network approach to fault diagnosis. Doctoral Thesis, Tampere University of Technology Publications 153.Google Scholar
- 30.Wu, T., Lin, C., & Weng, R. (2004). Probability estimates for multi-class classification by pairwise coupling. Journal of Machine Learning and Research, 5(Aug.), 975–1005.Google Scholar
- 31.Zaman, F., & Hirose, H. (2009). A new double bagging via the support vector machine with application to the condition diagnosis for the electric power apparatus. International Conference on Data Mining and Applications (ICDMA’09) (pp. 654–660).Google Scholar