Semantic Relationships in Multi-modal Graphs for Automatic Image Annotation

  • Vassilios Stathopoulos
  • Jana Urban
  • Joemon Jose
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4956)


It is important to integrate contextual information in order to improve the inaccurate results of current approaches for automatic image annotation. Graph based representations allow incorporation of such information. However, their behaviour has not been studied in this context. We conduct extensive experiments to show the properties of such representations using semantic relationships as a type of contextual information. We also experimented with different similarity measures for semantic features and results are presented.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE TPAMI 22(12), 1349–1380 (2000)Google Scholar
  2. 2.
    Smeaton, A.F., Over, P., Kraaij, W.: Evaluation campaigns and trecvid. In: MIR 2006, Santa Barbara, CA, USA, pp. 321–330. ACM Press, New York (2006)CrossRefGoogle Scholar
  3. 3.
    Wang, Y., Gong, S.: Refining image annotation using contextual relations between words. In: CIVR, Amsterdam, The Netherlands, ACM, pp. 425–432 (2007)Google Scholar
  4. 4.
    Pan, J.Y., Yang, H.J., Faloutsos, C., Duygulu, P.: Gcap: Graph-based automatic image captioning. In: CVPRW 2004, vol. 9, p. 146. IEEE Computer Society Press, Los Alamitos (2004)Google Scholar
  5. 5.
    Urban, J., Jose, J.M.: Adaptive image retrieval using a graph model for semantic feature integration. In: MIR 2006, Santa Barbara, CA, USA, pp. 117–126. ACM Press, New York (2006)CrossRefGoogle Scholar
  6. 6.
    Lova’sz, L.: Random walks on graphs: A survey. In: Combinatronics Bolyai Society for Mathematical Studies, Budapest, vol. 2, pp. 353–397 (1996)Google Scholar
  7. 7.
    Cilibrasi, R., Vitanyi, P.M.: Automatic meaning discovery using google. In: Hutter, M., Merkle, W., Vitanyi, P.M. (eds.) Kolmogorov Complexity and Applications. Number 06051 in Dagstuhl Seminar Proceedings, Schloss Dagstuhl, Germany, IBFI (2006)Google Scholar
  8. 8.
    Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. Addison Wesley, Reading (1999)Google Scholar
  9. 9.
    Duygulu, P., Barnard, K., de Freitas, J.F.G., Forsyth, D.A.: Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 97–112. Springer, Heidelberg (2002)Google Scholar
  10. 10.
    Yavlinsky, A., Schofield, E.J., Rüger, S.: Automated image annotation using global features and robust nonparametric density estimation. In: Leow, W.-K., Lew, M., Chua, T.-S., Ma, W.-Y., Chaisorn, L., Bakker, E.M. (eds.) CIVR 2005. LNCS, vol. 3568, pp. 507–517. Springer, Heidelberg (2005)Google Scholar
  11. 11.
    Jeon, J., Lavrenko, V., Manmatha, R.: Automatic image annotation and retrieval using cross-media relevance models. In: SIGIR 2003, Toronto, Canada, pp. 119–126. ACM Press, New York (2003)CrossRefGoogle Scholar
  12. 12.
    Barnard, K., Duygulu, P., Forsyth, D., de Freitas, N., Blei, D.M., Jordan, M.I.: Matching words and pictures. JMLR 3, 1107–1135 (2003)zbMATHCrossRefGoogle Scholar
  13. 13.
    Page, L., Brin, S., Motwani, R., Winograd, T.: The pagerank citation ranking: Bringing order to the web. Technical report, Stanford Digital Library Technologies Project (1998),
  14. 14.
    Kemeny, J.G., Snell, J.L.: Finite Markov Chains. Springer, New York (1960)zbMATHGoogle Scholar
  15. 15.
    Gallager, R.G.: Discrete Stochastic Processes. Kluwer Academic, Boston (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Vassilios Stathopoulos
    • 1
  • Jana Urban
    • 1
  • Joemon Jose
    • 1
  1. 1.Department of Computer ScienceUniversity of GlasgowGlasgowUK

Personalised recommendations