Skip to main content

Application of the Cross-Entropy Method to Dual Lagrange Support Vector Machine

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNAI,volume 5678)

Abstract

In this paper, cross entropy method is used for solving dual Lagrange support vector machine (SVM). Cross entropy (CE) method is a new practical approach which is widely used in some applications such as combinatorial optimization, learning algorithm and simulation. Our approach refers to Kernel Adatron which is solving dual Lagrange SVM using gradient ascent method. Hereby, the cross entropy method is applied to solve dual Lagrange SVM optimization problem to find the optimal or at least near optimal Lagrange multipliers as a solution. As known, the standard SVM with quadratic programming solver suffers from high computational time. Some real world datasets are used to test the algorithms and compare to the existing approach in terms of computation time and accuracy. Our approach is fast and produce good results in terms of generalization error.

Keywords

  • Cross entropy
  • generalization error
  • kernel adatron
  • Lagrange
  • Support vector machine
  • computation time

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-03348-3_61
  • Chapter length: 8 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   149.00
Price excludes VAT (USA)
  • ISBN: 978-3-642-03348-3
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   189.00
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Chin, K.K.: Support Vector Machines applied to Speech Pattern Classification, Dissertation, Darwin College, University of Cambridge (1998)

    Google Scholar 

  2. Kroese, D.P., Rubinstein, R.Y., Taimre, T.: Application of the Cross-Entropy Method to Clustering and Vector Quantization. Journal of Machine Learning Research (2004)

    Google Scholar 

  3. Frieb, T.T., Christianini, N., Campbell, C.: The Kernel Adatron Algorithm: a Fast and Simple Learning Procedure for Support Vector Machines. In: Shavlik, J. (ed.) Proceedings of the 15th International Conference on Machine Learning, pp. 188–196. Morgan Kaufmann, San Francisco (1998)

    Google Scholar 

  4. Hsu, C.W., Lin, C.J.: A Simple Decomposition Method for Support Vector Machines  46(1-3), 291–314 (2002)

    Google Scholar 

  5. Joachims, T.: Making large-scale support vector machine learning practical. In: Advances in kernel methods: support vector learning. MIT Press, Cambridge (1999)

    Google Scholar 

  6. Kecman, V., Vogt, M., Huang, T.M.: On the Equality of Kernel AdaTron and Sequential Minimal Optimization in Classification and Regression Tasks and Alike Algorithms for Kernel Machines. In: ESANN 2003 proceedings, Belgium, pp. 215–222. d-side publi. (2003)

    Google Scholar 

  7. Mannor, S., Peleg, D., Rubinstein, R.Y.: The cross entropy method for classification. In: Proceedings of the 22nd International Conference on Machine Learning, Bonn, Germany (2005)

    Google Scholar 

  8. Platt, J.C.: Fast training of support vector machines using sequential minimal optimization. In: Advances in kernel methods: support vector learning, pp. 185–208. MIT Press, Cambridge (1999)

    Google Scholar 

  9. Rubinstein, R.Y.: Optimization of computer simulation models with rare events. European Journal of Operations Research 99, 89–112 (1997)

    CrossRef  Google Scholar 

  10. Rubinstein, R., Kroese, D.: The cross-entropy method: A unified approach to combinatorial optimization, Monte-Carlo simulation, and machine-learning. Springer, Heidelberg (2004)

    CrossRef  MATH  Google Scholar 

  11. UCI Repository (2009), http://www.ics.uci.edu/~mlearn/mlrepository.html

  12. Zhou, H., Wang, J., Qiu, Y.: Application of the Cross Entropy Method to the Credit Risk Assessment in an Early Warning System. In: International Symposiums on Information Processing (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Santosa, B. (2009). Application of the Cross-Entropy Method to Dual Lagrange Support Vector Machine. In: Huang, R., Yang, Q., Pei, J., Gama, J., Meng, X., Li, X. (eds) Advanced Data Mining and Applications. ADMA 2009. Lecture Notes in Computer Science(), vol 5678. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03348-3_61

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-03348-3_61

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-03347-6

  • Online ISBN: 978-3-642-03348-3

  • eBook Packages: Computer ScienceComputer Science (R0)