A Corpus-Based Relevance Feedback Approach to Cross-Language Image Retrieval

  • Yih-Chen Chang
  • Wen-Cheng Lin
  • Hsin-Hsi Chen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4022)


This paper regards images with captions as a cross-media parallel corpus, and presents a corpus-based relevance feedback approach to combine the results of visual and textual runs. Experimental results show that this approach performs well. Comparing with the mean average precision (MAP) of the initial visual retrieval, the MAP is increased from 8.29% to 34.25% after relevance feedback from cross-media parallel corpus. The MAP of cross-lingual image retrieval is increased from 23.99% to 39.77% if combining the results of textual run and visual run with relevance feedback. Besides, the monolingual experiments also show the consistent effects of this approach. The MAP of monolingual retrieval is improved from 39.52% to 50.53% when merging the results of the text and image queries.


Image Retrieval Text Description Relevance Feedback Mean Average Precision CBIR System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Goodrum, A.A.: Image Information Retrieval: An Overview of Current Research. Information Science 3(2), 63–66 (2000)Google Scholar
  2. 2.
    Eidenberger, H., Breiteneder, C.: Semantic Feature Layers in Content-based Image Retrieval: Implementation of Human World Features. In: Proceedings of International Conference on Control, Automation, Robotic and Vision (2002)Google Scholar
  3. 3.
    Besançon, R., Hède, P., Moellic, P.A., Fluhr, C.: Cross-Media Feedback Strategies: Merging Text and Image Information to Improve Image Retrieval. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 709–717. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  4. 4.
    Jones, G.J.F., Groves, D., Khasin, A., Lam-Adesina, A., Mellebeek, B., Way, A.: Dublin City University at CLEF 2004: Experiments with the ImageCLEF St. Andrew’s Collection. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 653–663. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  5. 5.
    Baan, J., van Ballegooij, A., Geusenbroek, J.M., den Hartog, J., Hiemstra, D., List, J., Patras, I., Raaijmakers, S., Snoek, C., Todoran, L., Vendrig, J., de Vries, A., Westerveld, T., Worring, M.: Lazy Users and Automatic Video Retrieval Tools in the Lowlands. In: Proceedings of the Tenth Text REtrieval Conference, pp. 159–168 (2002)Google Scholar
  6. 6.
    Mori, Y., Takahashi, H., Oka, R.: Image-to-Word Transformation Based on Dividing and Vector Quantizing Images with Words. In: Proceedings of the First International Workshop on Multimedia Intelligent Storage and Retrieval Management (1999)Google Scholar
  7. 7.
    Duygulu, P., Barnard, K., Freitas, N., Forsyth, D.: Object Recognition as Machine Transla-tion: Learning a Lexicon for a Fixed Image Vocabulary. In: Proceedings of Seventh European Conference on Computer Vision, vol. 4, pp. 97–112 (2002)Google Scholar
  8. 8.
    Shi, J., Malik, J.: Normalized Cuts and Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8), 888–905 (2000)CrossRefGoogle Scholar
  9. 9.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society, Series B 39(1), 1–38 (1977)MATHMathSciNetGoogle Scholar
  10. 10.
    Jeon, J., Lavrenko, V., Manmatha, R.: Automatic Image Annotation and Retrieval using Cross-Media Relevance Models. In: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 119–126 (2003)Google Scholar
  11. 11.
    Lavrenko, V., Manmatha, R., Jeon, J.: A Model for Learning the Semantics of Pictures. In: Proceedings of the Seventeenth Annual Conference on Neural Information Processing Systems (2003)Google Scholar
  12. 12.
    Lin, W.C., Chang, Y.C., Chen, H.H.: Integrating Textual and Visual Information for Cross-Language Image Retrieval. In: Lee, G.G., Yamada, A., Meng, H., Myaeng, S.-H. (eds.) AIRS 2005. LNCS, vol. 3689, pp. 454–466. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  13. 13.
    Squire, D.M., Müller, W., Müller, H., Raki, J.: Content-Based Query of Image Databases, Inspirations from Text Retrieval: Inverted Files, Frequency-Based Weights and Relevance Feedback. In: Scandinavian Conference on Image Analysis, pp. 143–149 (1999)Google Scholar
  14. 14.
    Robertson, S.E., Walker, S., Beaulieu, M.: Okapi at TREC-7: Automatic Ad Hoc, Filtering, VLC and Interactive. In: Proceedings of the Seventh Text REtrieval Conference, National Institute of Standards and Technology, pp. 253–264 (1998)Google Scholar
  15. 15.
    Lin, W.C., Chang, Y.C., Chen, H.H.: From Text to Image: Generating Visual Query for Image Retrieval. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 664–675. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  16. 16.
    Clough, P., Müller, H., Deselaers, T., Grubinger, M., Lehmann, T.M., Jensen, J., Hersh, W.: The CLEF 2005 Cross–Language Image Retrieval Track. In: Peters, C., Gey, F.C., Gonzalo, J., Müller, H., Jones, G.J.F., Kluck, M., Magnini, B., de Rijke, M., Giampiccolo, D. (eds.) CLEF 2005. LNCS, vol. 4022, pp. 535–557. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  17. 17.
    Feng, K.M., Chen, H.H.: Effects of Positive and Negative Information Needs on Information Retrieval. Bulletin of the College of Engineering, National Taiwan University 90, 35–42 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Yih-Chen Chang
    • 1
  • Wen-Cheng Lin
    • 2
  • Hsin-Hsi Chen
    • 1
  1. 1.Department of Computer Science and Information EngineeringNational Taiwan UniversityTaipeiTaiwan
  2. 2.Department of Medical InformaticsTzu Chi UniversityHualienTaiwan

Personalised recommendations