Bootstrap Based Pattern Selection for Support Vector Regression
Support Vector Machine (SVM) results in a good generalization performance by employing the Structural Risk Minimization (SRM) principle. However, one drawback is O(n 3) training time complexity. In this paper, we propose a pattern selection method designed specifically for Support Vector Regression (SVR). In SVR training, only a few patterns called support vectors are used to construct the regression model while other patterns are not used at all. The proposed method tries to select patterns which are likely to become support vectors. With multiple bootstrap samples, we estimate the likelihood of each pattern to become a support vector. The proposed method automatically determines the appropriate number of patterns selected by estimating the expected number of support vectors. Through the experiments involving twenty datasets, the proposed method resulted in the best accuracy among the competing methods.
Unable to display preview. Download preview PDF.
- 2.Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press, Cambridge (2000)Google Scholar
- 3.Drucker, H., Burges, C.J.C., Kaufman, L., Smola, A., Vapnik, V.: Support Vector Regression Machines. In: Mozer, M.C., Jordan, M.I., Petsche, T. (eds.) Advances in Neural Information Processing System, vol. 9. MIT Press, Cambridge (1997)Google Scholar
- 4.Platt, J.C.: Fast Training of Support Vector Machines Using Sequential Minimal Optimization. In: Advanced in Kernel Methods; Support Vector Machines, pp. 185–208. MIT Press, Cambridge (1999)Google Scholar
- 5.Almeida, M.B., Braga, A., Braga, J.P.: SVM-KM: Speeding SVMs Learning with a Priori Cluster Selection and k-Means. In: Proc. of the 6th Brazilian Symposium on Neural Networks, pp. 162–167 (2000)Google Scholar
- 7.Bakir, G.H., Bottou, L., Weston, J.: Breaking SVM Complexity with Cross-Training. In: Advances in Neural Information Processing Systems, vol. 17, pp. 81–88 (2005)Google Scholar
- 8.Joachims, T.: Training Linear SVMs in Linear Time. In: Proc. of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 217–226 (2006)Google Scholar
- 10.Sun, J., Cho, S.: Pattern Selection for Support Vector Regression based on Sparsity and Variability. In: 2006 IEEE International Joint Conference on Neural Networks (IJCNN), pp. 559–602 (2006)Google Scholar