Skip to main content

Increasing Efficiency of SVM by Adaptively Penalizing Outliers

  • Conference paper
Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR 2005)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 3757))

Abstract

In this paper, a novel training method is proposed to increase the classification efficiency of support vector machine (SVM). The efficiency of the SVM is determined by the number of support vectors, which is usually large for representing a highly convoluted separation hypersurface. We noted that the separation hypersurface is made unnecessarily over-convoluted around extreme outliers, which dominate the objective function of SVM. To suppress the domination from extreme outliers and thus relatively simplify the shape of separation hypersurface, we propose a method of adaptively penalizing the outliers in the objective function. Since our reformulated objective function has the similar format of the standard SVM, the idea of the existing SVM training algorithms is borrowed for training the proposed SVM. Our proposed method has been tested on the UCI machine learning repository, as well as a real clinical problem, i.e., tissue classification in prostate ultrasound images. Experimental results show that our method is able to dramatically increase the classification efficiency of the SVM, without losing its generalization ability.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Vapnik, V.N.: The Natural of Statistical Learning Theory. Springer, New York (1995)

    Google Scholar 

  2. Burges, C.J.C.: A Tutorial on Support Vector Machines for Pattern Recognition. Data Mining and Knowledge Discovery 2, 121–167 (1998)

    Article  Google Scholar 

  3. Osuna, E., Freund, R., Giosi, F.: Training Support Vector Machines: An Application to Face Detection. In: Proc. IEEE. Conf. Computer Vision and Pattern Recognition, pp. 130–136 (1997)

    Google Scholar 

  4. Joachims, T.: A statistical learning learning model of text classification for support vector machines. In: Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, New Orleans (2001)

    Google Scholar 

  5. Zhan, Y., Shen, D.: Automated Segmentation of 3D US Prostate Images Using Statistical Texture-Based Matching Method. In: Ellis, R.E., Peters, T.M. (eds.) MICCAI 2003. LNCS, vol. 2878, pp. 688–696. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  6. Wan, V., Renals, S.: Speaker Verification using Sequence Discriminant Support Vector Machines. IEEE Transactions on Speech and Audio Processing 13(2), 203–210 (2005)

    Article  Google Scholar 

  7. Brown, M.P.S., Grundy, W.N., Lin, D., Cristianini, N., Sugnet, C.W., Furey, T.S., Ares Jr., M., Haussler, D.: Knowledge-based analysis of microarray gene expression data by using support vector machines. Genetics 97(1), 262–267 (2000)

    Google Scholar 

  8. Davatzikos, C., Shen, D., Lao, Z., Xue, Z., Karacali, B.: Morphological classification of medical images using nonlinear support vector machines. In: IEEE International Symposium on Biomedical Imaging (ISBI), Arlington, VA, April 15-18 (2004)

    Google Scholar 

  9. Lecun, Y., Jackel, L., Bottou, L., Brunot, A., Cortes, C., Denker, J., Drunker, H., Guyon, I., Muller, U., Sackinger, E., Simard, P., Vapnik, V.: Comparison of Learning Algorithms for Handwritten Digit Recognition. In: International Conference on Artificial Neural Networks, pp. 53–60 (1995)

    Google Scholar 

  10. Tian, Y.-L., Brown, L., Hampapur, A., Pankanti, S., Senior, A.W., Bolle, R.M.: Real World Real-time Automatic Recognition of Facial Expressions. In: IEEE workshop on performance evaluation of tracking and surveillance, Graz, Austria, March 31 (2003)

    Google Scholar 

  11. Osuna, E., Girosi, F.: Reducing the run-time complexity of Support Vector Machines, ICPR, Brisbane, Australia (1998)

    Google Scholar 

  12. Scholkopf, B., Smola, A.J.: Learning with Kernels. The MIT Press, Cambridge (2002)

    Google Scholar 

  13. Burges, C.J.C.: Simplified support vector decision rules. In: Proceedings of the 13th International Conference on Machine Learning, pp. 71–77 (1996)

    Google Scholar 

  14. Lee, Y.-J., Mangasarian, O.L.: RSVM: Reduced support vector machines. In: Proceedings of the First SIA International Conference on Data Mining (2001)

    Google Scholar 

  15. Hettich, S., Blake, C.L., Merz, C.J.: Repository of machine learning databases (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html

  16. Manjunath, B.S., Ma, W.Y.: Texture Features for Browsing and Retrieval of Image Data. IEEE Trans. on Pattern Anal. Mach. Intell. 18, 837–842 (1996)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2005 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Zhan, Y., Shen, D. (2005). Increasing Efficiency of SVM by Adaptively Penalizing Outliers. In: Rangarajan, A., Vemuri, B., Yuille, A.L. (eds) Energy Minimization Methods in Computer Vision and Pattern Recognition. EMMCVPR 2005. Lecture Notes in Computer Science, vol 3757. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11585978_35

Download citation

  • DOI: https://doi.org/10.1007/11585978_35

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-30287-2

  • Online ISBN: 978-3-540-32098-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics