Abstract
The Adaptive Multiple-hyperplane Machine (AMM) was recently proposed to deal with large-scale datasets. However, it has no principle to tune the complexity and sparsity levels of the solution. Addressing the sparsity is important to improve learning generalization, prediction accuracy and computational speedup. In this paper, we employ the max-margin principle and sparse approach to propose a new Sparse AMM (SAMM). We solve the new optimization objective function with stochastic gradient descent (SGD). Besides inheriting the good features of SGD-based learning method and the original AMM, our proposed Sparse AMM provides machinery and flexibility to tune the complexity and sparsity of the solution, making it possible to avoid overfitting and underfitting. We validate our approach on several large benchmark datasets. We show that with the ability to control sparsity, the proposed Sparse AMM yields superior classification accuracy to the original AMM while simultaneously achieving computational speedup.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
All datasets can be download at the URL http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets.
References
Aiolli, F., Sperduti, A.: Multiclass classification with multi-prototype support vector machines. J. Mach. Learn. Res. 6, 817–850 (2005)
Boser, B.E., Guyon, I.M., Vapnik, V.: A training algorithm for optimal margin classifiers. In: Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, pp. 144–152. ACM Press (1992)
Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 1–27 (2011)
Crammer, K., Singer, Y.: On the algorithmic implementation of multiclass kernel-based vector machines. J. Mach. Learn. Res. 2, 265–292 (2002)
Jaakkola, T., Meila, M., Jebara, T.: Maximum entropy discrimination. Technical report, Cambridge, MA, USA (1999)
Schapire, R.E., Freund, Y., Bartlett, P., Lee, W.S.: Boosting the margin: a new explanation for the effectiveness of voting methods. Ann. Stat. 26(5), 1651–1686 (1998)
Shalev-Shwartz, S., Kakade, S.M.: Mind the duality gap: logarithmic regret algorithms for online optimization. In: Advances in Neural Information Processing Systems, pp. 1457–1464 (2009)
Shalev-Shwartz, S., Singer, Y.: Logarithmic regret algorithms for strongly convex repeated games. The Hebrew University (2007)
Shalev-Shwartz, S., Singer, Y., Srebro, N., Cotter, A.: Pegasos: primal estimated sub-gradient solver for SVM. Math. Program. 127(1), 3–30 (2011)
Tsochantaridis, I.: Support vector machine learning for interdependent and structured output spaces. Ph.D. thesis, Providence, RI, USA, AAI3174684 (2005)
Vapnik, V.N.: Statistical Learning Theory. Wiley-Interscience, New York (1998)
Wang, Z., Djuric, N., Crammer, K., Vucetic, S.: Trading representability for scalability: adaptive multi-hyperplane machine for nonlinear classification. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 24–32. ACM, New York (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Nguyen, K., Le, T., Nguyen, V., Phung, D. (2016). Sparse Adaptive Multi-hyperplane Machine. In: Bailey, J., Khan, L., Washio, T., Dobbie, G., Huang, J., Wang, R. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2016. Lecture Notes in Computer Science(), vol 9651. Springer, Cham. https://doi.org/10.1007/978-3-319-31753-3_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-31753-3_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-31752-6
Online ISBN: 978-3-319-31753-3
eBook Packages: Computer ScienceComputer Science (R0)