Abstract
This paper deals with investigating l1-loss and l2-loss l2-regularized Support Vector Machines implemented in PermonSVM – a part of our PERMON toolbox. The loss functions quantify error between predicted and correct classifications of samples in cases of non-perfectly linearly separable classifications. In numerical experiments, we study properties of Hessians related to performance score of models and analyze convergence rate on 4 public available datasets. The Modified Proportioning and Reduced Gradient Projection algorithm is used as a solver for the dual Quadratic Programming problem resulting from Support Vector Machines formulations.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Libsvm data: Classification (binary class). https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary.html
Balay, S., Abhyankar, S., Adams, M.F., Brown, J., Brune, P., Buschelman, K., Eijkhout, V., Gropp, W.D., Kaushik, D., Knepley, M.G., McInnes, L.C., Rupp, K., Smith, B.F., Zhang, H.: PETSc – Portable, Extensible Toolkit for Scientific Computation. http://www.mcs.anl.gov/petsc
Brown, M., Grundy, W., Lin, D., Cristianini, N., Sugnet, C., Furey, T., Ares Jr., M., Haussler, D.: Knowledge-based analysis of microarray gene expression data by using support vector machines. Proc. Nat. Acad. Sci. U.S.A. 97(1), 262–267 (2000)
Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)
Dostál, Z., Pospíšil, L.: Minimizing quadratic functions with semidefinite hessian subject to bound constraints. Comput. Math. Appl. 70(8), 2014–2028 (2015)
Dostál, Z.: Optimal Quadratic Programming Algorithms, with Applications to Variational Inequalities, vol. 23. SOIA. Springer, New York (2009)
Foody, G.M., Mathur, A.: The use of small training sets containing mixed pixels for accurate hard image classification: training on mixed spectral responses for classification by a SVM. Remote Sens. Environ. 103(2), 179–189 (2006)
Hapla, V., Horák, D., Pecha, M.: PermonSVM (2017). http://permon.vsb.cz/permonsvm.htm
Hapla, V., Horák, D., Čermák, M., Kružík, J., Pospíšil, L., Sojka, R.: PermonQP (2015). http://permon.it4i.cz/qp/
Hapla, V., et al.: PERMON (Parallel, Efficient, Robust, Modular, Object-oriented, Numerical). http://permon.vsb.cz
Hsieh, C.J., Chang, K.W., Lin, C.J., Keerthi, S.S., Sundararajan, S.: A dual coordinate descent method for large-scale linear SVM. In: Proceedings of the 25th International Conference on Machine Learning, pp. 408–415. ACM (2008)
Kružík, J., Pecha, M., Hapla, V., Horák, D., Čermák, M.: Investigating Convergence of Linear SVM Implemented in PermonSVM Employing MPRGP Algorithm. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11087 LNCS, pp. 115–129 (2018)
Pecha, M., Hapla, V., Horák, D., Čermák, M.: Notes on the preliminary results of a linear two-class classifier in the permon toolbox. In: AIP Conference Proceedings, vol. 1978 (2018)
Shi, J., Lee, W.J., Liu, Y., Yang, Y., Wang, P.: Forecasting power output of photovoltaic systems based on weather classification and support vector machines. IEEE Trans. Ind. Appl. 48(3), 1064–1069 (2012)
Vishnu, A., Narasimhan, J., Holder, L., Kerbyson, D., Hoisie, A.: Fast and accurate support vector machines on large scale systems. In: 2015 IEEE International Conference on Cluster Computing, pp. 110–119, September 2015
Acknowledgments
This work has been supported by The Ministry of Education, Youth and Sports from the National Programme of Sustainability (NPU II) project IT4Innovations excellence in science - LQ1602; by Grant of SGS No. SP2018/165, VŠB - Technical University of Ostrava, Czech Republic and by the grant of the Czech Science Foundation (GACR) project no. GA17-22615S. The work has been also performed under the Project HPC-EUROPA3 (INFRAIA-2016-1-730897), with the support of the EC Research Innovation Action under the H2020 Programme; in particular, the author gratefully acknowledges the support of School of Mathematics, The University of Edinburgh, United Kingdom and the computer resources and technical support provided by Edinburgh Parallel Computing Centre (EPCC).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Pecha, M., Horák, D. (2020). Analyzing l1-loss and l2-loss Support Vector Machines Implemented in PERMON Toolbox. In: Zelinka, I., Brandstetter, P., Trong Dao, T., Hoang Duy, V., Kim, S. (eds) AETA 2018 - Recent Advances in Electrical Engineering and Related Sciences: Theory and Application. AETA 2018. Lecture Notes in Electrical Engineering, vol 554. Springer, Cham. https://doi.org/10.1007/978-3-030-14907-9_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-14907-9_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-14906-2
Online ISBN: 978-3-030-14907-9
eBook Packages: EngineeringEngineering (R0)