Multimedia Tools and Applications

, Volume 75, Issue 20, pp 12627–12644 | Cite as

Multi-view multi-label learning for image annotation

  • Fuhao Zou
  • Yu LiuEmail author
  • Hua Wang
  • Jingkuan Song
  • Jie Shao
  • Ke Zhou
  • Sheng Zheng


Image annotation is posed as multi-class classification problem. Pursuing higher accuracy is a permanent but not stale challenge in the field of image annotation. To further improve the accuracy of image annotation, we propose a multi-view multi-label (abbreviated by MVML) learning algorithm, in which we take multiple feature (i.e., view) and ensemble learning into account simultaneously. By doing so, we make full use of the complementarity among the views and the base learners of ensemble learning, leading to higher accuracy of image annotation. With respect to the different distribution of positive and negative training examples, we propose two versions of MVML: the Boosting and Bagging versions of MVML. The former is suitable for learning over balanced examples while the latter applies to the opposite scenario. Besides, the weights of base learner is evaluated on validation data instead of training data, which will improve the generalization ability of the final ensemble classifiers. The experimental results have shown that the MVML is superior to the ensemble SVM of single view.


Image Annotation Ensemble learning Multi-view learning Multi-label learning 



This work is supported in part by the National Basic Research Program (973 Program) of China under Grant No. 2011CB302305, the National Natural Science Foundation of China under Grant No. 61232004. The authors appreciate the valuable suggestions from the anonymous reviewers and the Editors.


  1. 1.
    Alham NK, Li M, Liu Y, Ponraj M, Qi M (2012) A distributed SVM ensemble for image classification and annotation, 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), IEEE, pp 1581–1584Google Scholar
  2. 2.
    Bauer E, Kohavi R (1999) An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning, Springer 36(1-2):105–139CrossRefGoogle Scholar
  3. 3.
    Breiman L (1996) Bagging predictors. Mach Learn, Springer 24 (2):123–140MathSciNetzbMATHGoogle Scholar
  4. 4.
    Breiman Leo V (2001) Random forests. Mach Learn 45 (1):5–32MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Burges C JC (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Disc, Springer 5 (2):121–167CrossRefGoogle Scholar
  6. 6.
    Chang C-C, Lin C-J (2011) LIBSVM: A library for support vector machines. ACM Trans Intell Syst Technol 2 (3):1–27CrossRefGoogle Scholar
  7. 7.
    Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297zbMATHGoogle Scholar
  8. 8.
    Dietterich TG (2000) Ensemble methods in machine learning, Multiple classifier systems, pp 1–15Google Scholar
  9. 9.
    Dietterichl TGS (2002) Ensemble learning. The handbook of brain theory and neural networks, pp 405–408Google Scholar
  10. 10.
    Freund Y, Schapire RE (1995) A desicion-theoretic generalization of on-line learning and an application to boosting, Computational learning theory. Springer, pp 23–37Google Scholar
  11. 11.
    Galar M, Alberto F, Tartas EB, Sola HB, Herrera F (2012) A review on ensembles for the class imbalance problem: Bagging-, Boosting-, and Hybrid-Based approaches. IEEE Transactions on Systems, Man, and Cybernetics, Part C, IEEE, pp 463–484Google Scholar
  12. 12.
    Gonen M, Alpayd E (2011) Multiple kernel learning algorithms. The Journal of Machine Learning Research., pp 2211–2268Google Scholar
  13. 13.
    Haykin S (2004) A comprehensive neural networks. Neural Netw 2 (2004)Google Scholar
  14. 14.
    Hosmer DW, Lemeshow S, Sturdivant RX (2000) Introduction to the logistic regression model, Wiley Online Library Inc.Google Scholar
  15. 15.
    Khoshgoftaar TM, Van Hulse J, Napolitano A (2011) Comparing boosting and bagging techniques with noisy and imbalanced data. IEEE Transactions on Systems, Man, and Cybernetics, Part A, pp 552–568Google Scholar
  16. 16.
    Kim H-C, Pang S, Je H-M, Kim D, Bang S-Y (2002) Support vector machine ensemble with bagging, Pattern recognition with support vector machines. Springer, pp 397–408Google Scholar
  17. 17.
    Muda Z (2007) Classification and image annotation for bridging the semantic gap. In: Proceedings of the summer school on multimedia semantics, vol 2007, pp 15–21Google Scholar
  18. 18.
    Sewell M (2008) Ensemble learning. RN, Citeseer 11(2):1–15Google Scholar
  19. 19.
    Song J, Yang Y, Huang Z, Shen HT, Hong R (2011) Multiple feature hashing for real-time large scale near-duplicate video retrieval. In: Proceedings of the 19th ACM international conference on multimedia, pp 423–432Google Scholar
  20. 20.
    Valentini G, Dietterich TG (2003) Low bias bagged support vector machines, ICML, pp 752–759Google Scholar
  21. 21.
    Wolpert DH (1992) Stacked generalization. Neural Netw Elsevier 5(2):241–259MathSciNetCrossRefGoogle Scholar
  22. 22.
    Xu X-S, Xue X, Zhou Z-H (2011) Ensemble multi-instance multi-label learning approach for video annotation task. In: Proceedings of the 19th ACM international conference on multimedia. ACM, pp 1153–1156Google Scholar
  23. 23.
    Yan G, Ma G, Zhu L (2006) Support vector machines ensemble based on fuzzy integral for classification. Advances in Neural Networks-ISNN 2006. Springer, pp 974–980Google Scholar
  24. 24.
    Yan G, Ma G, Zhu L (2006) Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE, pp 1088–1099Google Scholar
  25. 25.
    Yang Y, Huang Z, Yang Y, Liu J, Shen HT, Luo J (2013) Local image tagging via graph regularized joint group sparsity. Pattern Recogn, Elsevier Sc Inc 46(5):1358–1368CrossRefzbMATHGoogle Scholar
  26. 26.
    Yang Y, Yang Y, Huang Z, Shen HT (2011) Tag localization with spatial correlations and joint group sparsity. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 881–888Google Scholar
  27. 27.
    Yang Y, Yang Y, Shen HT, Zhang Y, Du X, Zhou X (2013) Discriminative nonnegative spectral clustering with out-of-sample extension. IEEE Trans Data Knowl Eng (TKDE) 25(8):1760–1771CrossRefGoogle Scholar
  28. 28.
    Yang Y, Zha Z-J, Gao Y, Zhu X, Chua T-S (2014) Exploiting web images for semantic video indexing via robust sample-speci?c loss. IEEE Trans Multimed 16(6):1677–1689CrossRefGoogle Scholar
  29. 29.
    Zhang L, Gao Y, Xia Y, Dai Q, Li X (2014) A fine-grained image categorization system by cellet-encoded spatial pyramid modeling. IEEE Transactions on Industrial Electronics, pp 1–8Google Scholar
  30. 30.
    Zhang L, Han Y, Yang Y, Song M, Yan S, Tian Q (2013) Discovering discriminative graphlets for aerial image categories recognition. IEEE Transactions on Image Processing, pp 5071–5084Google Scholar
  31. 31.
    Zhang L, Song M, Zhao Q, Liu X, Bu J, Chen C (2013) IEEE, probabilistic graphlet transfer for photo cropping. IEEE Trans Image Process 22 (2):802–815MathSciNetCrossRefGoogle Scholar
  32. 32.
    Zhang L, Yi Y, Gao Y, Yu Y, Wang C, Li X (2014) A probabilistic associative model for segmenting weakly supervised images. IEEE Trans Image Process 23(9):4150–4159MathSciNetCrossRefGoogle Scholar
  33. 33.
    Zhang L, Xia Y, Ji R, Li O (2014) IEEE, Spatial-aware object-level saliency prediction by learning graphlet hierarchies. IEEE Trans Ind Electron 99:1–8Google Scholar
  34. 34.
    Zhou Z-H (2009) Ensemble learning. Encyclopedia of Biometrics. Springer, pp 270–273Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Fuhao Zou
    • 1
  • Yu Liu
    • 2
    Email author
  • Hua Wang
    • 2
  • Jingkuan Song
    • 3
  • Jie Shao
    • 4
  • Ke Zhou
    • 2
  • Sheng Zheng
    • 2
  1. 1.School of Computer Science and TechnologyHuazhong University of Science and TechnologyWuhanChina
  2. 2.Wuhan National Laboratory for OptoelectronicsHuazhong University of Science and TechnologyWuhanChina
  3. 3.Department of Information Engineering and Computer ScienceUniversity of TrentoTrentoItaly
  4. 4.School of Computer Science and TechnologyUniversity of Electronic Science and Technology of ChinaWuhanChina

Personalised recommendations