ADMA 2009: Advanced Data Mining and Applications pp 595-602 | Cite as
Application of the Cross-Entropy Method to Dual Lagrange Support Vector Machine
Abstract
In this paper, cross entropy method is used for solving dual Lagrange support vector machine (SVM). Cross entropy (CE) method is a new practical approach which is widely used in some applications such as combinatorial optimization, learning algorithm and simulation. Our approach refers to Kernel Adatron which is solving dual Lagrange SVM using gradient ascent method. Hereby, the cross entropy method is applied to solve dual Lagrange SVM optimization problem to find the optimal or at least near optimal Lagrange multipliers as a solution. As known, the standard SVM with quadratic programming solver suffers from high computational time. Some real world datasets are used to test the algorithms and compare to the existing approach in terms of computation time and accuracy. Our approach is fast and produce good results in terms of generalization error.
Keywords
Cross entropy generalization error kernel adatron Lagrange Support vector machine computation timePreview
Unable to display preview. Download preview PDF.
References
- 1.Chin, K.K.: Support Vector Machines applied to Speech Pattern Classification, Dissertation, Darwin College, University of Cambridge (1998)Google Scholar
- 2.Kroese, D.P., Rubinstein, R.Y., Taimre, T.: Application of the Cross-Entropy Method to Clustering and Vector Quantization. Journal of Machine Learning Research (2004)Google Scholar
- 3.Frieb, T.T., Christianini, N., Campbell, C.: The Kernel Adatron Algorithm: a Fast and Simple Learning Procedure for Support Vector Machines. In: Shavlik, J. (ed.) Proceedings of the 15th International Conference on Machine Learning, pp. 188–196. Morgan Kaufmann, San Francisco (1998)Google Scholar
- 4.Hsu, C.W., Lin, C.J.: A Simple Decomposition Method for Support Vector Machines 46(1-3), 291–314 (2002)Google Scholar
- 5.Joachims, T.: Making large-scale support vector machine learning practical. In: Advances in kernel methods: support vector learning. MIT Press, Cambridge (1999)Google Scholar
- 6.Kecman, V., Vogt, M., Huang, T.M.: On the Equality of Kernel AdaTron and Sequential Minimal Optimization in Classification and Regression Tasks and Alike Algorithms for Kernel Machines. In: ESANN 2003 proceedings, Belgium, pp. 215–222. d-side publi. (2003)Google Scholar
- 7.Mannor, S., Peleg, D., Rubinstein, R.Y.: The cross entropy method for classification. In: Proceedings of the 22nd International Conference on Machine Learning, Bonn, Germany (2005)Google Scholar
- 8.Platt, J.C.: Fast training of support vector machines using sequential minimal optimization. In: Advances in kernel methods: support vector learning, pp. 185–208. MIT Press, Cambridge (1999)Google Scholar
- 9.Rubinstein, R.Y.: Optimization of computer simulation models with rare events. European Journal of Operations Research 99, 89–112 (1997)CrossRefGoogle Scholar
- 10.Rubinstein, R., Kroese, D.: The cross-entropy method: A unified approach to combinatorial optimization, Monte-Carlo simulation, and machine-learning. Springer, Heidelberg (2004)CrossRefMATHGoogle Scholar
- 11.UCI Repository (2009), http://www.ics.uci.edu/~mlearn/mlrepository.html
- 12.Zhou, H., Wang, J., Qiu, Y.: Application of the Cross Entropy Method to the Credit Risk Assessment in an Early Warning System. In: International Symposiums on Information Processing (2008)Google Scholar