Skip to main content
SpringerLink
Log in

Multi-class Bagged Proximal Support Vector Machines for the ImageNet Challenging Problem

  • Conference paper
  • First Online:
Future Data and Security Engineering (FDSE 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 13076))

Included in the following conference series:

Abstract

We propose the new multi-class of bagged proximal support vector machines (MC-Bag-PSVM) for handling the ImageNet challenging problem with very large number of images and a thousand classes. Our MC-Bag-PSVM trains in the parallel manner ensemble binary PSVM classifiers used for the One-Versus-All (OVA) multi-class strategy on multi-core computer with GPUs. The binary PSVM model is constructed by bagged binary PSVM models built in under-sampling training dataset. The numerical test results on ILSVRC 2010 dataset show that our MC-Bag-PSVM algorithm is faster and more accurate than the state-of-the-art linear SVM algorithm. An example of its effectiveness is given with an accuracy of 75.64% obtained in the classification of ImageNet-1000 dataset having 1,261,405 images in 2048 deep features into 1,000 classes in 29.5 min using a PC Intel(R) Core i7-4790 CPU, 3.6 GHz, 4 cores and Gigabyte GeForce RTX 2080Ti 11 GB GDDR6, 4352 CUDA cores.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bosch, A., Zisserman, A., Muñoz, X.: Scene classification Via pLSA. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 517–530. Springer, Heidelberg (2006). https://doi.org/10.1007/11744085_40

    Chapter  Google Scholar 

  2. Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)

    MathSciNet  MATH  Google Scholar 

  3. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(27), 1–27 (2011)

    Article  Google Scholar 

  4. Chollet, F.: Xception: deep learning with depthwise separable convolutions. CoRR abs/1610.02357 (2016)

    Google Scholar 

  5. Deng, J., Berg, A.C., Li, K., Fei-Fei, L.: What does classifying more than 10,000 image categories tell us? In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6315, pp. 71–84. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15555-0_6

    Chapter  Google Scholar 

  6. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Li, F.F.: ImageNet: a large-scale hierarchical image database. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)

    Google Scholar 

  7. Do, T.-N.: Parallel multiclass stochastic gradient descent algorithms for classifying million images with very-high-dimensional signatures into thousands classes. Vietnam J. Comput. Sci. 1(2), 107–115 (2014). https://doi.org/10.1007/s40595-013-0013-2

    Article  Google Scholar 

  8. Do, T.-N., Nguyen, V.-H., Poulet, F.: Speed up SVM algorithm for massive classification tasks. In: Tang, C., Ling, C.X., Zhou, X., Cercone, N.J., Li, X. (eds.) ADMA 2008. LNCS (LNAI), vol. 5139, pp. 147–157. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88192-6_15

    Chapter  Google Scholar 

  9. Do, T.-N., Poulet, F.: Parallel multiclass logistic regression for classifying large scale image datasets. In: Le Thi, H.A., Nguyen, N.T., Do, T.V. (eds.) Advanced Computational Methods for Knowledge Engineering. AISC, vol. 358, pp. 255–266. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-17996-4_23

    Chapter  Google Scholar 

  10. Do, T.-N., Tran-Nguyen, M.-T.: Incremental parallel support vector machines for classifying large-scale multi-class image datasets. In: Dang, T.K., Wagner, R., Küng, J., Thoai, N., Takizawa, M., Neuhold, E. (eds.) FDSE 2016. LNCS, vol. 10018, pp. 20–39. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48057-2_2

    Chapter  Google Scholar 

  11. Doan, T.-N., Do, T.-N., Poulet, F.: Large scale classifiers for visual classification tasks. Multimed. Tools Appl. 74(4), 1199–1224 (2014). https://doi.org/10.1007/s11042-014-2049-4

    Article  Google Scholar 

  12. Fan, R.E., Chang, K.W., Hsieh, C.J., Wang, X.R., Lin, C.J.: LIBLINEAR: a library for large linear classification. J. Mach. Learn. Res. 9(4), 1871–1874 (2008)

    MATH  Google Scholar 

  13. Fung, G., Mangasarian, O.L.: Proximal support vector machine classifiers. In: Proceedings of the seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 26–29 August 2001, pp. 77–86 (2001)

    Google Scholar 

  14. Fung, G., Mangasarian, O.L.: Multicategory proximal support vector machine classifiers. Mach. Learn. 59(1–2), 77–97 (2005)

    Article  Google Scholar 

  15. He, H., Garcia, E.A.: Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 21(9), 1263–1284 (2009)

    Article  Google Scholar 

  16. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR abs/1512.03385 (2015)

    Google Scholar 

  17. Herrero-Lopez, S., Williams, J.R., Sanchez, A.: Parallel multiclass classification using SVMs on GPUs. In: Proceedings of the 3rd Workshop on General-Purpose Computation on Graphics Processing Units, pp. 2–11. ACM, New York (2010)

    Google Scholar 

  18. Hido, S., Kashima, H.: Roughly balanced bagging for imbalanced data. In: SIAM International Conference on Data Mining, pp. 143–152 (2008)

    Google Scholar 

  19. Japkowicz, N. (ed.): AAAI’Workshop on Learning from Imbalanced Data Sets. No. WS-00-05 in AAAI Tech Report (2000)

    Google Scholar 

  20. Kreßel, U.H.G.: Pairwise classification and support vector machines. In: Schölkopf, B., Burges, C.J.C., Smola, A.J. (eds.) Advances in Kernel Methods, pp. 255–268. MIT Press, Cambridge (1999)

    Google Scholar 

  21. Lenca, P., Lallich, S., Do, T.-N., Pham, N.-K.: A comparison of different off-centered entropies to deal with class imbalance for decision trees. In: Washio, T., Suzuki, E., Ting, K.M., Inokuchi, A. (eds.) PAKDD 2008. LNCS (LNAI), vol. 5012, pp. 634–643. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-68125-0_59

    Chapter  Google Scholar 

  22. Li, F., Perona, P.: A Bayesian hierarchical model for learning natural scene categories. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), 20–26 June 2005, San Diego, CA, USA, pp. 524–531 (2005)

    Google Scholar 

  23. Lowe, D.: Object recognition from local scale invariant features. In: Proceedings of the 7th International Conference on Computer Vision, pp. 1150–1157 (1999)

    Google Scholar 

  24. Lowe, D.: Distinctive image features from scale invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)

    Article  Google Scholar 

  25. OpenMP Architecture Review Board: OpenMP application program interface version 3.0 (2008). http://www.openmp.org/mp-documents/spec30.pdf

  26. Perronnin, F., Sánchez, J., Liu, Y.: Large-scale image categorization with explicit data embedding. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2297–2304 (2010)

    Google Scholar 

  27. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014)

    Google Scholar 

  28. Sivic, J., Zisserman, A.: Video Google: a text retrieval approach to object matching in videos. In: 9th IEEE International Conference on Computer Vision (ICCV 2003), 14–17 October 2003, Nice, France, pp. 1470–1477 (2003)

    Google Scholar 

  29. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. CoRR abs/1512.00567 (2015)

    Google Scholar 

  30. Vapnik, V.: The Nature of Statistical Learning Theory. Springer, Heidelberg (1995). https://doi.org/10.1007/978-1-4757-2440-0

    Book  MATH  Google Scholar 

  31. Whaley, R., Dongarra, J.: Automatically tuned linear algebra software. In: Ninth SIAM Conference on Parallel Processing for Scientific Computing (1999). cD-ROM Proceedings

    Google Scholar 

  32. Wu, J.: Power mean SVM for large scale visual classification. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2344–2351 (2012)

    Google Scholar 

Download references

Acknowledgments

This work has received support from the College of Information Technology, Can Tho University. We would like to thank very much the Big Data and Mobile Computing Laboratory.

Author information

Authors and Affiliations

  1. College of Information Technology, Can Tho University, Cantho, 92000, Vietnam

    Thanh-Nghi Do

  2. UMI UMMISCO 209 (IRD/UPMC), Sorbonne University, Pierre and Marie Curie University, Paris 6, France

    Thanh-Nghi Do

Authors
  1. Thanh-Nghi Do
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thanh-Nghi Do .

Editor information

Editors and Affiliations

  1. HCMC University of Technology, Ho Chi Minh City, Vietnam

    Tran Khanh Dang

  2. Johannes Kepler University of Linz, Linz, Austria

    Josef Küng

  3. Sungkyunkwan University, Suwon, Korea (Republic of)

    Tai M. Chung

  4. Hosei University, Koganei-shi, Tokyo, Japan

    Makoto Takizawa

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Do, TN. (2021). Multi-class Bagged Proximal Support Vector Machines for the ImageNet Challenging Problem. In: Dang, T.K., Küng, J., Chung, T.M., Takizawa, M. (eds) Future Data and Security Engineering. FDSE 2021. Lecture Notes in Computer Science(), vol 13076. Springer, Cham. https://doi.org/10.1007/978-3-030-91387-8_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-91387-8_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-91386-1

  • Online ISBN: 978-3-030-91387-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation