Efficient Large Scale Image Classification via Prediction Score Decomposition

  • Duy-Dinh LeEmail author
  • Tien-Dung Mai
  • Shin’ichi Satoh
  • Thanh Duc Ngo
  • Duc Anh Duong
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9910)


There has been growing interest in reducing the test time complexity of multi-class classification problems with large numbers of classes. The key idea to solve it is to reduce the number of classifier evaluations used to predict labels. The state-of-the-art methods usually employ the label tree approach that usually suffers the well-know error propagation problem and it is difficult for parallelization for further speedup. We propose another practical approach, with the same goal of using a small number of classifiers to achieve a good trade-off between testing efficiency and classification accuracy. The proposed method analyzes the correlation among classes, suppresses redundancy, and generates a small number of classifiers that best approximate the prediction scores of the original large number of classes. Different from label-tree methods in which each test example follows a different traversing path from the root to a leaf node and results in a different set of classifiers each time, the proposed method applies the same set of classifiers to all test examples. As a result, it is much more efficient in practice, even in the case of using the same number of classifier evaluations as the label-tree methods. Experiments on several large datasets including ILSVRC2010-1K, SUN-397, and Caltech-256 show the efficiency of our method.


Large scale classification Label tree Matrix decomposition 



This research is funded by Vietnam National University Ho Chi Minh City (VNU-HCM) under grant number B2015-26-01.

Supplementary material

419981_1_En_46_MOESM1_ESM.pdf (52 kb)
Supplementary material 1 (pdf 51 KB)


  1. 1.
    Akata, Z., Perronnin, F., Harchaoui, Z., Schmid, C.: Good practice in large-scale learning for image classification. PAMI 36(3), 507–520 (2013)CrossRefGoogle Scholar
  2. 2.
    Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing multi-class to binary: a unifying approach for margin classifiers. J. Mach. Learn. Res. 1, 113–141 (2001)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Amit, Y., Fink, M., Srebro, N., Ullman, S.: Uncovering shared structures in multiclass classification. In: ICML (2007)Google Scholar
  4. 4.
    Bengio, S., Weston, J., Grangier, D.: Label embedding trees for large multi-class task. In: NIPS (2010)Google Scholar
  5. 5.
    Beygelzimer, A., Langford, J., Lifshits, Y., Sorkin, G., Strehl, A.: Conditional probability tree estimation analysis and algorithms. In: UAI (2009)Google Scholar
  6. 6.
    Beygelzimer, A., Langford, J., Ravikumar, P.: Error-correcting tournaments. In: Gavaldà, R., Lugosi, G., Zeugmann, T., Zilles, S. (eds.) ALT 2009. LNCS (LNAI), vol. 5809, pp. 247–262. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-04414-4_22 CrossRefGoogle Scholar
  7. 7.
    Chen, Y., Crawford, M., Ghosh, J.: Integrating support vector machines in a hierarchical output space decomposition framework. In: IGARSS (2004)Google Scholar
  8. 8.
    Crammer, K., Singer, Y.: On the learnability and design of output codes for multiclass problems. Mach. Learn. 47(2–3), 201–233 (2002)CrossRefzbMATHGoogle Scholar
  9. 9.
    Deng, J., Dong, W., Socher, R., Li, L., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: CVPR (2009)Google Scholar
  10. 10.
    Deng, J., Satheesh, S., Berg, A., Fei-Fei, L.: Fast and balanced: efficient label tree learning for large scale object recognition. In: NIPS (2011)Google Scholar
  11. 11.
    Denton, E.L., Zaremba, W., Bruna, J., Lecun, Y., Fergus, R.: Exploiting linear structure within convolutional networks for efficient evaluation. In: NIPS (2014)Google Scholar
  12. 12.
    Dietterich, T.G., Bakiri, G.: Solving multi-class learning problems via error-correcting output codes. J. Artif. Intell. Res. 2, 263–286 (1995)zbMATHGoogle Scholar
  13. 13.
    Escalera, S., Pujol, O., Radeva, P.: Error-correcting ouput codes library. J. Mach. Learn. Res. 11, 661–664 (2010)Google Scholar
  14. 14.
    Escalera, S., Tax, M., Pujol, O., Radeva, P.: Subclass problem-dependent design for error-correcting output codes. PAMI (2008)Google Scholar
  15. 15.
    Fan, R.E., Chang, K.W., Hsieh, C.J., Wang, X.R., Lin, C.J.: Liblinear: a library for large linear classification. J. Mach. Learn. Res. 8, 1871–1874 (2008)zbMATHGoogle Scholar
  16. 16.
    Gao, T., Koller, D.: Discriminative learning of relaxed hierarchy for large-scale visual recognition. In: ICCV (2011)Google Scholar
  17. 17.
    Griffin, G., Holub, A., Perona, P.: Caltech-256 object category dataset. Technical report, California Institute of Technology (2007)Google Scholar
  18. 18.
    Griffin, G., Perona, P.: Learning and using taxonomies for fast visual categorization. In: CVPR (2008)Google Scholar
  19. 19.
    Harchaoui, Z., Douze, M., Paulin, M., Dudik, M., Malick, J.: Large-scale image classification with trace-norm regularization. In: CVPR (2012)Google Scholar
  20. 20.
    Jaderberg, M., Vedaldi, A., Zisserman, A.: Speeding up convolutional neural networks with low rank expansions. In: BMVC (2014)Google Scholar
  21. 21.
    Kusakunniran, W., Satoh, S., Zhang, J., Wu, Q.: Attribute-based learning for large scale object classification. In: ICME (2013)Google Scholar
  22. 22.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: CVPR (2006)Google Scholar
  23. 23.
    Lin, Y., Lv, F., Zhu, S., Yang, M., Cour, T., Yu, K., Cao, L., Huang, T.: Large-scale image classification: fast feature extraction and svm training. In: CVPR (2011)Google Scholar
  24. 24.
    Liu, B., Sadeghi, F., Tappen, M., Shamir, O., Liu, C.: Probabilistic label trees for efficient large scale image classification. In: CVPR (2013)Google Scholar
  25. 25.
    Liu, S., Yi, H., Chia, L.T., Rajan, D.: Adaptive hierarchical multi-class svm classifier for texture-based image classification. In: ICME (2005)Google Scholar
  26. 26.
    Loeff, N., Farhadi, A.: Scene discovery by matrix factorization. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5305, pp. 451–464. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-88693-8_33 CrossRefGoogle Scholar
  27. 27.
    Marszałek, M., Schmid, C.: Constructing category hierarchies for visual recognition. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5305, pp. 479–491. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-88693-8_35 CrossRefGoogle Scholar
  28. 28.
    Ng, A., Jordan, M., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: NIPS (2002)Google Scholar
  29. 29.
    Platt, J.C., Cristianini, N., Shawe-taylor, J.: Large margin dags for multi-class classification. In: NIPS (2000)Google Scholar
  30. 30.
    Pujol, O., Radeva, P., Vitrià, J.: Discriminant ECOC: A heuristic method for application dependent design of error correcting output codes. PAMI (2006)Google Scholar
  31. 31.
    Rifkin, R., Klautau, A.: In defense of one-vs-all classification. J. Mach. Learn. Res. 5, 101–141 (2004)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Song, H.O., Girshick, R., Zickler, S., Geyer, C., Felzenszwalb, P., Darrell, T.: Generalized sparselet models for real-time multiclass object recognition. PAMI (2013)Google Scholar
  33. 33.
    Sun, M., Huang, W., Savarese, S.: Find the best path: an efficient and accurate classifier for image hierarchies. In: ICCV (2013)Google Scholar
  34. 34.
    Vedaldi, A., Fulkerson, B.: VLFeat - an open and portable library of computer vision algorithms. In: ACM International Conference on Multimedia (2010)Google Scholar
  35. 35.
    Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., Gong, Y.: Locality-constrained linear coding for image classification. In: CVPR (2010)Google Scholar
  36. 36.
    Xia, S., Li, J., Xia, L., Ju, C.: Tree-structured support vector machines for multi-class classification. In: Liu, D., Fei, S., Hou, Z., Zhang, H., Sun, C. (eds.) ISNN 2007. LNCS, vol. 4493, pp. 392–398. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-72395-0_50 CrossRefGoogle Scholar
  37. 37.
    Xiao, J., Hays, J., Ehinger, K., Oliva, A., Torralba, A.: Sun database: large-scale scene recognition from abbey to zoo. In: CVPR (2010)Google Scholar
  38. 38.
    Yuan, X., Lai, W., Mei, T., Hua, X., Wu, X., Li, S.: Automatic video genre categorization using hierarchical svm. In: ICIP (2006)Google Scholar
  39. 39.
    Zhang, X., Liang, L., Shum, H.: Spectral error correcting output codes for efficient multiclass recognition. In: ICCV (2009)Google Scholar
  40. 40.
    Zhao, B., Xing, E.P.: Sparse output coding for large-scale visual recognition. In: CVPR (2013)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Duy-Dinh Le
    • 1
    • 2
    Email author
  • Tien-Dung Mai
    • 1
  • Shin’ichi Satoh
    • 2
  • Thanh Duc Ngo
    • 1
  • Duc Anh Duong
    • 1
  1. 1.University of Information Technology, VNU-HCMHo Chi Minh CityVietnam
  2. 2.National Institute of InformaticsTokyoJapan

Personalised recommendations