Advertisement

Double SVMSBagging: A Subsampling Approach to SVM Ensemble

  • Faisal ZamanEmail author
  • Hideo Hirose
Chapter
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 52)

Abstract

In ensemble methods, pooling the decisions of multiple unstable classifiers often lead to improvements in the generalization performance substantially in many applications. We propose here a new ensemble method, Double SVMSBagging, which is a variant of double bagging. In this method we have used subsampling in order to make the out-of-bag samples larger and trained support vector machine as the additional classifier on these out-of-bag samples. The underlying base classifier is the decision tree. We have used radial basis function kernel, expecting that the new classifier can perform efficiently in both linear and non-linear feature space. We have studied the performance of the proposed ensemble method in several benchmark datasets with different subsampling rate (SSR). We have applied the proposed method in partial discharge classification of the gas insulated switchgear (GIS). We compare the performance of double SVMsbagging with other well-known classifier ensemble methods in condition diagnosis; the double SVMsbagging performed better than other ensemble method in this case. We applied the double SVMsbagging in 15 UCI benchmark datasets and compare its accuracy with other ensemble methods, e.g., Bagging, Adaboost, Random Forest and Rotation Forest. The performance of this method with optimum SSR generate significantly lower prediction error than Rotation Forest and Adaboost for most of the datasets.

Keywords

Support vector machine Double bagging Subsampling rate CART Partial discharge analysis 

References

  1. 1.
    Blake. C., & Merz, C. (1999). UCI repository of machine learning databases, http://www.ics.uci.edu/mlearn/MLRepository.html.
  2. 2.
    Breiman, L., Friedman, J., Olshen, R., & Stone, C. (1984). Classification and regression trees. Belmont, CA: Wadsworth.zbMATHGoogle Scholar
  3. 3.
    Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2), 123–140.MathSciNetzbMATHGoogle Scholar
  4. 4.
    Breiman, L. (1998). Arcing classifiers. Annals of Statistics, 26(3), 801–849.MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.zbMATHCrossRefGoogle Scholar
  6. 6.
    Breiman, L. (2001). Statistical modeling: the two cultures. Statistical Science, 16(3), 199–231 (with discussion).Google Scholar
  7. 7.
    Burges, C. (1998). A tutorial on support vector machines for pattern recognition, Data Mining and Knowledge Discovery, 2, 121–167.CrossRefGoogle Scholar
  8. 8.
    Chang, C., & Lin, C. (2001). LIBSVM: a library for support vector machines, software available at http://www.csie.ntu.edu.tw/cjlin/libsvm.
  9. 9.
    Cortes, C., & Vapnik, V. (1995). support-vector networks, Machine Learning, 20, 273–297.zbMATHGoogle Scholar
  10. 10.
    Freund Y., & Schapire, R. (1996). Experiments with a new boosting algorithm. Proceedings of the thirteenth international conference machine learning (pp. 148–156). San Francisco, MA: Morgan Kaufmann.Google Scholar
  11. 11.
    Friedman, J., Hastie, T., & Tibshirani, R. (2000). Additive logistic regression: a statistical view of boosting. Annals of Statistics, 28, 337–407(with discussion).Google Scholar
  12. 12.
    Gestel, T., Suykens, J., Baesens, B., Viaene, S., Vanthienen, J., Dedene, G., Moor, B., & Vandewalle, J. (2001). Bench marking least squares support vector machine classifiers, Machine Learning, 54(1), 5–32.CrossRefGoogle Scholar
  13. 13.
    Gulski, E. (1995). Digital analysis of partial discharges. IEEE Transactions on Dielectrics and Electrical Insulation, 2(5), 822–837.CrossRefGoogle Scholar
  14. 14.
    Hirose, H., Matsuda, S., & H., Hikita. (2006). Electrical insulation diagnosing using a new statistical classification method. Proceedings of the 8th international conference on properties and applications of dielectric materials (ICPADM2006) (pp. 698–701).Google Scholar
  15. 15.
    Hirose, H., Hikita, M., Ohtsuka, S., Tsuru, S., & Ichimaru, J. , (2008). Diagnosing the electric power apparatuses using the decision tree method. IEEE Transactions on Dielectrics and Electrical Insulation, 15(5), 1252–1261.CrossRefGoogle Scholar
  16. 16.
    Hirose, H., Zaman, F., Tsuru, K., Tsuboi, T., & Okabe, S. (2008). Diagnosis accuracy in electric power apparatuses conditions using the classification methods. IEICE Technical Report, 108(243), 39–44.Google Scholar
  17. 17.
    Hothorn, T., & Lausen, B. (2003). Double-bagging: combining classifiers by bootstrap aggregation, Pattern Recognition, 36(6), 1303–1309.zbMATHCrossRefGoogle Scholar
  18. 18.
    Iba, W., & Langley, R. (1992). Induction of one-level decision trees. Proceedings of the nineteenth international conference on machine learning, Aberdeen, Scotland.Google Scholar
  19. 19.
    Joachims, T. (1999). Making large-scale support vector machine learning practical. Advances in kernel methods: support vector machines (pp. 169–184). Cambridge, MA: MIT Press.Google Scholar
  20. 20.
    Li, Y., Cal, Y., Yin, R., & Xu, X. (2005). Fault diagnosis based on support vector machine ensemble. Proceedings of the 2005 international conference on machine Learning. Cybernet, 6, 3309–3314.Google Scholar
  21. 21.
    Lin, H., Lin, C., & Wen, R. (2007). A Note on Platt’s Probabilistic outputs for support vector machines, Machine Learning, 68(3), 267–276.CrossRefGoogle Scholar
  22. 22.
    Meyer, D., Leisch, F., & Hornik, K. (2003). The support vector machine under test. Neurocomputing, 55, 169–186.CrossRefGoogle Scholar
  23. 23.
    Patton, R., Lopez-Toribio, C., & Uppal, F. (1999). Artificial intelligence approaches to fault diagnosis, condition monitoring. IEE Colloquium on Machinery, External Structures and Health (Ref. No. 1999/034), 5/1–5/18.Google Scholar
  24. 24.
    Platt, J. (1999). Fast training of support vector machines using sequential minimal optimization. In B. Scholkopf, C.J.C. Burges, & A.J. Smola (Eds.), Advances Kernel methods – support vector learning (pp. 185–208). Cambridge, MA: MIT.Google Scholar
  25. 25.
    Platt, J. (2000). Probabilistic outputs for support vector machines and comparison to regularized likelihood methods. In A.J. Smola, P. Bartlett, B. Scholkopf, & D. Schuurmans, (Eds.), Advances in large margin classifiers (pp. 61–74). Cambridge, MA: MIT Press.Google Scholar
  26. 26.
    Rodríguez, J., Kuncheva, L., & Alonso. C. (2006). Rotation forest: a new classifier ensemble method. PAMI, 28(10), 1619–1630.CrossRefGoogle Scholar
  27. 27.
    Scholkopf, B., Smola, A., & Muller, K. (1998). Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10(5), 1299–1319.CrossRefGoogle Scholar
  28. 28.
    Scholkopf, B., Burges, C., & Smola, A. (1999). Introduction to support vector learning. In B. Scholkopf, C.J.C. Burges, & A.J. Smola (Eds.), Advances in kernel methods: support vector learning (pp. 1–15). Cambridge, MA: MIT Press.Google Scholar
  29. 29.
    Sorsa, T. (1995). Neural network approach to fault diagnosis. Doctoral Thesis, Tampere University of Technology Publications 153.Google Scholar
  30. 30.
    Wu, T., Lin, C., & Weng, R. (2004). Probability estimates for multi-class classification by pairwise coupling. Journal of Machine Learning and Research, 5(Aug.), 975–1005.Google Scholar
  31. 31.
    Zaman, F., & Hirose, H. (2009). A new double bagging via the support vector machine with application to the condition diagnosis for the electric power apparatus. International Conference on Data Mining and Applications (ICDMA’09) (pp. 654–660).Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2009

Authors and Affiliations

  1. 1.Department of Systems Design and InformaticsKyushu Institute of TechnologyKyushuJapan

Personalised recommendations