Skip to main content
SpringerLink
Log in

Parallel incremental power mean SVM for the classification of large-scale image datasets

  • Regular Paper
  • Published:
International Journal of Multimedia Information Retrieval Aims and scope Submit manuscript

Abstract

The amount of image data becomes larger and larger, both image size (due the higher resolution) and image number. It is estimated for personal use only, an average single user will take 100,000 images during his life. The growth of image data is illustrated by the dataset size, for example ImageNet benchmark dataset is made of more than 14 million images and more than 21,000 classes. This is very challenging for classification algorithms. They have to deal with time and space complexity and very imbalanced data when using SVM algorithms. We present extensions of Power Mean SVM to deal with such data. The first one is an incremental version to deal with the space complexity, the second one is a parallel version of the incremental version to deal with time complexity and the last one is the use of a balanced bagging algorithm for training binary classifiers to deal with imbalanced data. We evaluate our parallel incremental version of balanced bagging PmSVM on the 1,000 classes of ImageNet (ILSVRC 2010). The results show that our algorithm can be run on standard PC (with eg. 2 or 4 GB RAM); it is 255 times faster than the original version and 1,276 times faster than state-of-the-art linear classifier, LIBLINEAR with 80 cores.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Berg A, Deng J, Li FF (2010) Large scale visual recognition challenge 2010. Tech Rep. http://www.image-net.org/challenges/LSVRC/2010/index

  2. Chua TS, Tang J, Hong R, Li H, Luo Z, Zheng, YT (2009) Nus-wide: a real-world web image database from national university of Singapore. In: Proceedings of ACM Conference on Image and Video Retrieval (CIVR’09). Santorini, Greece

  3. Csurka G, Dance CR, Fan L, Willamowski J, Bray C (2004) Visual categorization with bags of keypoints. In: Workshop on Statistical Learning in Computer Vision, ECCV, pp 1–22

  4. Deng J, Berg AC, Li K, Li FF (2010) What does classifying more than 10, 000 image categories tell us? In: Daniilidis K, Maragos P, Paragios N (eds) ECCV, Part V. Lecture Notes in Computer Science, vol 6315. Springer pp 71–84

  5. Do TN, Nguyen VH, Poulet F (2008) Speed up SVM algorithm for massive classification tasks. In: Tang C, Ling CX, Zhou X, Cercone N, Li X (eds) ADMA. Lecture Notes in Computer Science, vol 5139. Springer

  6. Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A (2010) The Pascal visual object classes (VOC) challenge. Int J Comput Vis 88(2):303–338

    Article  Google Scholar 

  7. Griffin G, Holub A, Perona P (2007) Caltech-256 object category dataset. Tech. Rep. CNS-TR-2007-001. California Institute of Technology. http://authors.library.caltech.edu/7694

  8. Guermeur Y (2007) SVM multiclasses, théorie et applications

  9. Hsieh CJ, Chang KW, Lin CJ, Keerthi SS, Sundararajan S (2008) A dual coordinate descent method for large-scale linear SVM. In: International Conference on Machine Learning, pp 408–415

  10. Huiskes MJ, Thomee B, Lew MS (2010) New trends and ideas in visual concept detection: The mir Flickr retrieval evaluation initiative. In: Proceedings of the International Conference on Multimedia Information Retrieval, MIR ’10. ACM, New York, pp 527–536. doi:10.1145/1743384.1743475. http://doi.acm.org/10.1145/1743384.1743475

  11. Krebel UH-G (1999) Pairwise classification and support vector machines. In: Schölkopf B, Burges CJC, Smola AJ (eds) Advances in Kernel methods. MIT Press, Cambridge, pp 255–268

  12. Lenca P, Lallich S, Do TN, Pham NK (2008) A comparison of different off-centered entropies to deal with class imbalance for decision trees. In: The Pacific-Asia Conference on Knowledge Discovery and Data Mining, LNAI 5012. Springer, New York, pp 634–643

  13. Li FF, Fergus R, Perona P (2007) Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. Comput Vis Image Underst 106(1):59–70

    Google Scholar 

  14. Lin Y, Lv F, Zhu S, Yang M, Cour T, Yu K, Cao L, Huang TS (2011) Large-scale image classification: fast feature extraction and SVM training. In: CVPR. IEEE pp 1689–1696

  15. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110. http://dx.doi.org/10.1023/B:VISI.0000029664.99615.94

    Google Scholar 

  16. MPI-Forum.: MPI: a message-passing interface standard URL http://www.mpi-forum.org

  17. OpenMP Architecture Review Board: OpenMP application program interface version 3.0 (2008). http://www.openmp.org/mp-documents/spec30.pdf

  18. Perronnin F, Sánchez J, Liu Y (2010) Large-scale image categorization with explicit data embedding. In: CVPR. IEEE, pp 2297–2304

  19. Pham NK, Do TN, Lenca P, Lallich S (2008) Using local node information in decision trees: coupling a local decision rule with an off-centered entropy. In: International Conference on Data Mining. CSREA Press, Las Vegas, pp 117–123

  20. Platt J, Cristianini N, Shawe-Taylor J (2000) Large margin dags for multiclass classification. Adv Neural Inf Process Syst 12:547–553

    Google Scholar 

  21. Vapnik V (1995) The nature of statistical learning theory. Springer, New York

    Book  MATH  Google Scholar 

  22. Vedaldi A, Zisserman A (2012) Efficient additive kernels via explicit feature maps. IEEE Trans Pattern Anal Mach Intell 34(3):480–492

    Article  Google Scholar 

  23. Visa S, Ralescu A (2005) Issues in mining imbalanced data sets—a review paper. In: Midwest Artificial Intelligence and Cognitive Science Conference. Dayton, USA pp 67–73

  24. Weiss GM, Provost F (2003) Learning when training data are costly: the effect of class distribution on tree induction. J Artif Intell Res 19:315–354

    MATH  Google Scholar 

  25. Weston J, Watkins C (1999) Support vector machines for multi-class pattern recognition. In: Proceedings of the Seventh European Symposium on Artificial, Neural Networks, pp 219–224

  26. Wu J (2010) A fast dual method for hik svm learning. In: Daniilidis K, Maragos P, Paragios N (eds) European Conference on Computer Vision, Lecture Notes in Computer Science. Springer, New York, vol 6312, pp 552–565

  27. Wu J (2012) Power mean svm for large scale visual classification. In: CVPR. IEEE pp 2344–2351

  28. Wu J, Tan WC, Rehg JM (2011) Efficient and effective visual codebook generation using additive kernels. J Mach Learn Res 12:3097–3118

    MATH  MathSciNet  Google Scholar 

  29. Yu HF, Hsieh CJ, Chang KW, Lin CJ (2012) Large linear classification when data cannot fit in memory. TKDD 5(4):23

    Article  Google Scholar 

  30. Yuan GX, Ho CH, Lin CJ (2012) Recent advances of large-scale linear classification. Proc IEEE 100(9):2584–2603

    Article  Google Scholar 

Download references

Acknowledgments

This work was partially funded by Region Bretagne (France) and VIED (Vietnam International Education Development).

Author information

Authors and Affiliations

  1. IRISA, Université de Rennes 1, Campus Universitaire de Beaulieu, 35042 , Rennes Cedex, France

    Thanh-Nghi Doan & François Poulet

  2. College of Information Technology, Can Tho University, Can Tho, Vietnam

    Thanh-Nghi Do

Authors
  1. Thanh-Nghi Doan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thanh-Nghi Do
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. François Poulet
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thanh-Nghi Doan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Doan, TN., Do, TN. & Poulet, F. Parallel incremental power mean SVM for the classification of large-scale image datasets. Int J Multimed Info Retr 3, 89–96 (2014). https://doi.org/10.1007/s13735-014-0053-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13735-014-0053-0

Keywords

Search

Navigation