Robust Multi-view Manifold Ranking for Image Retrieval

  • Jun WuEmail author
  • Jianbo Yuan
  • Jiebo Luo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9652)


Graph-based similarity ranking plays a key role in improving image retrieval performance. Its current trend is to fuse the ranking results from multiple feature sets, including textual feature, visual feature and query log feature, to elevate the retrieval effectiveness. The primary challenge is how to effectively exploit the complementary properties of different features. Another tough issue is the highly noisy features contributed by users, such as textual tags and query logs, which makes the exploration of such complementary properties difficult. This paper proposes a Multi-view Manifold Ranking (M2R) framework, in which multiple graphs built on different features are integrated to simultaneously encode the similarity ranking. To deal with the high noise issue inherent in the user-contributed features, a data cleaning solution based on visual-neighbor voting is embedded into M2R, thus called Robust M2R (RM2R). Experimental results show that the proposed method significantly outperforms the existing approaches, especially when the user-contributed features are highly noisy.


Image retrieval Multi-view learning Manifold ranking Data cleaning 



The authors would like to thank the anonymous reviewers for their constructive suggestions. This work was supported in part by the ‘Natural Science Foundation of China’ (61301185, 61370070 and 61300071), the ‘Fundamental Research Funds for the Central Universities’ (2015JBM029), and the ‘Science Foundation of Beijing Jiaotong University’ (2015RC008).


  1. 1.
    Blum, A., Mitchell, T.: Combining labeled and unlabeled data with co-training. In: COLT, pp. 92–100 (1998)Google Scholar
  2. 2.
    Deng, C., Ji, R., Tao, D., Gao, X., Li, X.: Weakly supervised multi-graph learning for robust image reranking. IEEE Trans. Multimedia 16(3), 785–795 (2014)CrossRefGoogle Scholar
  3. 3.
    He, J., Li, M., Zhang, H., Tong, H., Zhang, C.S.: Manifold-ranking based image retrieval. In: ACM Multimedia, pp. 9–16 (2004)Google Scholar
  4. 4.
    Hoi, S., Liu, W., Chang, S.F.: Semi-supervised distance metric learning for collaborative image retrieval. In: CVPR, pp. 1–7 (2008)Google Scholar
  5. 5.
    Jain, V., Varma, M.: Learning to re-rank: query-dependent image re-ranking using click data. In: Proceedings of the 20th International Conference on World Wide Web, WWW 2011, pp. 277–286 (2011)Google Scholar
  6. 6.
    Li, X., Snoek, G.C., Worring, M.: Learning social tag relevance by neighbor voting. IEEE Trans. Multimedia 11(7), 1310–1322 (2009)CrossRefGoogle Scholar
  7. 7.
    Pan, Y., Yao, T., Mei, T., Li, H., Ngo, C.W., Rui, Y.: Click-through-based cross-view learning for image search. In: SIGIR, pp. 717–726 (2014)Google Scholar
  8. 8.
    Smith, G., Brien, C., Ashman, H.: Evaluating implicit judgments from image search clickthrough data. J. Am. Soc. Inf. Sci. Technol. 63(12), 2451–2462 (2012)CrossRefGoogle Scholar
  9. 9.
    Su, J., Huang, W., Yu, P., Tseng, V.: Efficient relevance feedback for content-based image retrieval by mining user navigation patterns. IEEE Trans. Knowl. Data Eng. 23(3), 360–372 (2011)CrossRefGoogle Scholar
  10. 10.
    Wang, B., Pan, F., Hu, K.M., Paul, J.C.: Manifold-ranking based retrieval using k-regular nearest neighbor graph. Pattern Recogn. 45(4), 1569–1577 (2012)CrossRefGoogle Scholar
  11. 11.
    Wang, M., Li, H., Tao, D., Lu, K., Wu, X.: Multimodal graph-based reranking for web image search. IEEE Trans. Image Process. 21(11), 4649–4661 (2012)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Wang, T., Dai, G., Ni, B., Xu, D., Siewe, F.: A distance measure between labeled combinatorial maps. Comput. Vis. Image Underst. 116(12), 1168–1177 (2012)CrossRefGoogle Scholar
  13. 13.
    Wang, W., Zhou, Z.-H.: Analyzing co-training style algorithms. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) ECML 2007. LNCS (LNAI), vol. 4701, pp. 454–465. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  14. 14.
    Wu, J., Lin, Z., Lu, M.Y.: Asymmetric semi-supervised boosting for SVM active learning in CBIR. In: ACM CIVR, pp. 182–188 (2010)Google Scholar
  15. 15.
    Wu, J., Shen, H., Li, Y.D., Xiao, Z.B., Lu, M.Y., Wang, C.L.: Learning a hybrid similarity measure for image retrieval. Pattern Recogn. 46(11), 2927–2939 (2013)CrossRefzbMATHGoogle Scholar
  16. 16.
    Wu, J., Li, Y., Feng, S., Shen, H.: A self-immunizing manifold ranking for image retrieval. In: Pei, J., Tseng, V.S., Cao, L., Motoda, H., Xu, G. (eds.) PAKDD 2013, Part II. LNCS, vol. 7819, pp. 426–436. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  17. 17.
    Wu, J., Lu, M.Y., Wang, C.L.: Collaborative learning between visual content and hidden semantic for image retrieval. In: ICDM, pp. 1133–1138 (2010)Google Scholar
  18. 18.
    Wu, J., Xiao, Z.B., Wang, H.S., Shen, H.: Learning with both unlabeled data and query logs for image search. Comput. Electr. Eng. 40(3), 964–973 (2014)CrossRefGoogle Scholar
  19. 19.
    Xu, B., Bu, J., Chen, C., Wang, C., Cai, D., He, X.: EMR: a efficient manifold ranking model for content-based image retrieval. IEEE Trans. Knowl. Data Eng. 27(1), 102–114 (2014)Google Scholar
  20. 20.
    Yang, X., Zhang, Y., Yao, T., Ngo, C.W., Mei, T.: Click-boosting multi-modality graph-based reranking for image search. Multimedia Syst. 21(2), 217–227 (2015)CrossRefGoogle Scholar
  21. 21.
    Yin, P.Y., Bhanu, B., Chang, K.C., Dong, A.: Integrating relevance feedback techniques for image retrieval using reinforcement learning. IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1536–1551 (2005)CrossRefGoogle Scholar
  22. 22.
    Yu, J., Tao, D., Wang, M., Rui, Y.: Learning to rank using user clicks and visual features for image retrieval. IEEE Trans. Cybern. 45(4), 767–779 (2015)CrossRefGoogle Scholar
  23. 23.
    Yu, J., Rui, Y., Chen, B.: Exploiting click constraints and multi-view features for image re-ranking. IEEE Trans. Multimedia 16(1), 159–168 (2014)CrossRefGoogle Scholar
  24. 24.
    Zelnik-Manor, L., Perona, P.: Self-tuning spectral clustering. In: NIPS, pp. 1601–1608 (2004)Google Scholar
  25. 25.
    Zhang, L., Rui, Y.: Image search: from thousands to billions in 20 years. ACM Trans. Multimedia Comput. Commun. Appl. 9(1s), 36:1–36:20 (2013)CrossRefGoogle Scholar
  26. 26.
    Zhang, Y., Yang, X., Mei, T.: Image search reranking with query-dependent click-based relevance feedback. IEEE Trans. Image Process. 23(10), 4448–4459 (2014)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Zhou, D., Weston, J., Gretton, A., Bousquet, O., Scholkopf, B.: Ranking on data manifolds. In: NIPS, pp. 169–176 (2003)Google Scholar
  28. 28.
    Zhou, X.S., Huang, T.S.: Relevance feedback in image retrieval: a comprehensive review. Multimedia Syst. 8(6), 536–544 (2003)CrossRefGoogle Scholar
  29. 29.
    Zhou, Z.H., Chen, K.J., Dai, H.B.: Enhancing relevance feedback in image retrieval using unlabeled data. ACM Trans. Inf. Syst. 24(2), 219–244 (2006)CrossRefGoogle Scholar
  30. 30.
    Zhou, Z.H., Zhan, D.C., Yang, Q.: Semi-supervised learning with very few labeled training examples. In: AAAI, pp. 675–680 (2007)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Beijing Key Lab of Traffic Data Analysis and MiningBeijing Jiaotong UniversityBeijingChina
  2. 2.Department of Computer ScienceUniversity of RochesterRochesterUSA

Personalised recommendations