Contextual Image Annotation via Projection and Quantum Theory Inspired Measurement for Integration of Text and Visual Features

  • Leszek Kaliciak
  • Jun Wang
  • Dawei Song
  • Peng Zhang
  • Yuexian Hou
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7052)

Abstract

Multimedia information retrieval suffers from the semantic gap, a difference between human perception and machine representation of images. In order to reduce the gap, a quantum theory inspired theoretical framework for integration of text and visual features has been proposed. This article is a follow-up work on this model. Previously, two relatively straightforward statistical approaches for making associations between dimensions of both feature spaces were employed, but with unsatisfactory results. In this paper, we propose to alleviate the problem regarding unannotated images by projecting them onto subspaces representing visual context and by incorporating a quantum-like measurement. The proposed principled approach extends the traditional vector space model (VSM) and seamlessly integrates with the tensor-based framework. Here, we experimentally test the novel association methods in a small-scale experiment.

Keywords

multimedia retrieval quantum theory image annotation tensor product 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Duygulu, P., Barnard, K., de Freitas, J.F.G., Forsyth, D.: Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, pp. 349–354. Springer, Heidelberg (2002)Google Scholar
  2. 2.
    Yang, J., Jiang, Y.G., Hauptmann, A.G., Ngo, C.W.: Evaluating Bag-of-Visual-Words Representations in Scene Classification. In: Proc. of the Int. Workshop on Multimedia IR, vol. 206 (2007)Google Scholar
  3. 3.
    Jamieson, M., Dickinson, S., Stevenson, S., Wachsmuth, S.: Using Language to Drive the Perceptual Grouping of Local Image Features. In: IEEE Comp. Society Conference on Comp. Vision and Pattern Rec., vol. 2, pp. 2102–2109 (2006)Google Scholar
  4. 4.
    Li, J., Wang, J.Z.: Real-Time Computerized Annotation of Pictures. IEEE Tran. on Pattern Anal. and Machine Int. 30, 985–1002 (2008)CrossRefGoogle Scholar
  5. 5.
    Yanai, K.: Generic Image Classification Using Visual Knowledge on the Web. In: Proc. of the 11-th ACM Int. Conf. on Multimedia, pp. 167–176 (2003)Google Scholar
  6. 6.
    Tjondronegoro, D., Zhang, J., Gu, J., Nguyen, A., Geva, S.: Integrating Text Retrieval and Image Retrieval in XML Document Searching. In: Advances in XML Inf. Retr. and Evaluation (2005)Google Scholar
  7. 7.
    Rahman, M.M., Bhattacharya, P., Desai, B.C.: A Unified Image Retrieval Framework on Local Visual and Semantic Concept-Based Feature Spaces. J. Visual Communication and Image Representation 20, 450–462 (2009)CrossRefGoogle Scholar
  8. 8.
    Simpson, M., Rahaman, M.M.: Text and Content Based Approaches to Image Retrieval for the ImageClef2009 Medical Retrieval Track. In: Working Notes for the CLEF 2009 Workshop (2009)Google Scholar
  9. 9.
    Min, P., Kazhdan, M., Funkhouser, T.: A comparison of text and shape matching for retrieval of online 3D models. In: Heery, R., Lyon, L. (eds.) ECDL 2004. LNCS, vol. 3232, pp. 209–220. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  10. 10.
    van Rijsbergen, C.J.: The Geometry of Information Retrieval. Cambridge University Press, Cambridge (2004)CrossRefMATHGoogle Scholar
  11. 11.
    Griffiths, R.B.: Consistent Quantum Theory. Cambridge University Press, Cambridge (2003)Google Scholar
  12. 12.
    Melucci, M.: Context Modeling and Discovery Using Vector Space Bases. In: Proc. of the ACM Conf. on Inf. and Knowledge Management, pp. 808–815 (2005)Google Scholar
  13. 13.
    Di Buccio, E., Melucci, M., Song, D.: Towards Predicting Relevance Using a Quantum-Like Framework. In: The 33rd European Conference on IR, pp. 19–21 (2011)Google Scholar
  14. 14.
    Biancalana, C., Lapolla, A., Micarelli, A.: Personalized web search using correlation matrix for query expansion. In: Cordeiro, J., Hammoudi, S., Filipe, J. (eds.) Web Information Systems and Technologies. LNBIP, vol. 18, pp. 186–198. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  15. 15.
    Aharonov, Y., Albert, D.Z., Au, C.K.: New Interpretation of the Scalar Product in Hilbert Space. Phys. Rev. Lett. 47, 1029–1031 (1981)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Wang, J., Song, D., Kaliciak, L.: Tensor Product of Correlated Text and Visual Features: A Quantum Theory Inspired Image Retrieval Framework. In: AAAI-Fall 2010 Symp. on Quant. Inf. for Cognitive, Social, and Semantic Processes, pp. 109–116 (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Leszek Kaliciak
    • 1
  • Jun Wang
    • 1
  • Dawei Song
    • 1
  • Peng Zhang
    • 1
  • Yuexian Hou
    • 2
  1. 1.The Robert Gordon UniversityAberdeenUK
  2. 2.Tianjin UniversityTianjinChina

Personalised recommendations