Abstract
Selecting the optimal parameters for support vector machine (SVM) has long been a hot research topic. Aiming for support vector classification/regression (SVC/SVR) with the radial basis function (RBF) kernel, we summarize the rough line rule of the penalty parameter and kernel width, and propose a novel linear search method to obtain these two optimal parameters. We use a direct-setting method with thresholds to set the epsilon parameter of SVR. The proposed method directly locates the right search field, which greatly saves computing time and achieves a stable, high accuracy. The method is more competitive for both SVC and SVR. It is easy to use and feasible for a new data set without any adjustments, since it requires no parameters to set.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ahn, J., 2010. A stable hyperparameter selection for the Gaussian RBF kernel for discrimination. Statist. Anal. Data Min., 3(3):142–148. [doi:10.1002/sam.10073]
Bengio, Y., 2000. Gradient-based optimization of hyper-parameters. Neur. Comput., 12(8):1889–1900. [doi:10. 1162/089976600300015187]
Bi, J., Bennett, K.P., 2003. A geometric approach to support vector regression. Neurocomputing, 55(1–2):79–108. [doi:10.1016/S0925-2312(03)00380-1]
Chapelle, O., Vapnik, V., Bousquet, O., Mukherjee, S., 2002. Choosing multiple parameters for support vector machines. Mach. Learn., 46(1/3):131–159. [doi:10.1023/A: 1012450327387]
Cherkassky, V., Ma, Y.Q., 2004. Practical selection of SVM parameters and noise estimation for SVM regression. Neur. Networks, 17(1):113–126. [doi:10.1016/S0893-6080(03)00169-2]
Cristianini, N., Kandola, J., Elisseeff, A., Shawe-Taylor, J., 2006. On kernel target alignment. Innov. Mach. Learn., 194:205–256. [doi:10.1007/3-540-33486-6_8]
Deng, N.Y., Tian, Y.G., 2004. A New Method in Data Mining—Support Vector Machine. Science Press, Beijing, China (in Chinese).
Duan, K., Keerthi, S.S., Poo, A.N., 2003. Evaluation of simple performance measures for tuning SVM hyperparameters. Neurocomputing, 51:41–59. [doi:10.1016/S0925-2312(02) 00601-X]
Gijsberts, A., Metta, G., Rothkrantz, L., 2010. Evolutionary optimization of least-squares support vector machines. Data Min., 8(4):277–297. [doi:10.1007/978-1-4419-1280-0_12]
Huang, C.L., Dun, J.F., 2008. A distributed PSO-SVM hybrid system with feature selection and parameter optimization. Appl. Soft Comput., 8(4):1381–1391. [doi:10.1016/j.asoc. 2007.10.007]
Huang, C.L., Wang, C.J., 2006. A GA-based feature selection and parameters optimization for support vector machines. Expert Syst. Appl., 31(2):231–240. [doi:10.1016/j.eswa. 2005.09.024]
Keerthi, S.S., Lin, C.J., 2003. Asymptotic behaviors of support vector machines with Gaussian kernel. Neur. Comput., 15(7):1667–1689. [doi:10.1162/089976603321891855]
Kim, D., Song, J., Lee, J., Choi, B., 2007. Support vector machine learning for region-based image retrieval with relevance feedback. ETRI J., 29(5):700–702. [doi:10. 4218/etrij.07.0207.0037]
Lau, K.W., Wu, Q.H., 2008. Local prediction of non-linear time series using support vector regression. Pattern Recogn., 41(5):1539–1547. [doi:10.1016/j.patcog.2007.08.013]
Po, L.M., Ma, W.C., 2002. A novel four-step search algorithm for fast block motion estimation. IEEE Trans. Circ. Syst. Video Technol., 6(3):313–317. [doi:10.1109/76.499840]
Roohi, M., Mirjalily, G., Sadeghi, M.T., 2007. Face Detection Using a Modified SVM-Based Classifier. 7th Int. Conf. on Computational Intelligence and Multimedia Applications, p.356–360. [doi:10.1109/ICCIMA.2007.243]
Sapankevych, N., Sankar, R., 2009. Time series prediction using support vector machines, a survey. IEEE Computat. Intell. Mag., 4(2):24–38. [doi:10.1109/MCI.2009.932254]
Scholkopf, B., Smola, A.J., 2002. Learning with Kernels. MIT Press, Cambridge.
Suykens, J.A.K., Vandewalle, J., de Moor, B., 2001. Optimal control by least squares support vector machines. Neur. Networks, 14(1):23–35. [doi:10.1016/S0893-6080(00)000 77-0]
Vapnik, V., Golowich, S.E., Smola, A., 1997. Support Vector Method for Function Approximation, Regression Estimation, and Signal Processing. Advances in Neural Information Processing Systems 9 — Proc. Conf. on Advances in Neural Information Processing Systems, p.281–287.
Vladimir, N., Vapnik, V., 2000. The Nature of Statistical Learning Theory. Springer Verlag, New York.
Wang, T.Y., Chiang, H.M., 2007. Fuzzy support vector machine for multi-class text categorization. Inform. Process. Manag., 43(4):914–929. [doi:10.1016/j.ipm.2006.09.011]
Zhu, S., Ma, K.K., 1997. A New Diamond Search Algorithm for Fast Block Matching Motion Estimation. Proc. Int. Conf. on Information, Communications and Signal Processing, 1:292–296. [doi:10.1109/ICICS.1997.647106]
Author information
Authors and Affiliations
Corresponding author
Additional information
Project supported by the National Basic Research Program (973) of China (No. 2009CB724006) and the National Natural Science Foundation of China (No. 60977010)
Rights and permissions
About this article
Cite this article
Pang, Hx., Dong, Wd., Xu, Zh. et al. Novel linear search for support vector machine parameter selection. J. Zhejiang Univ. - Sci. C 12, 885–896 (2011). https://doi.org/10.1631/jzus.C1100006
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/jzus.C1100006