Automatic Annotation of Images from the Practitioner Perspective

  • Peter G. B. Enser
  • Christine J. Sandom
  • Paul H. Lewis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3568)


This paper describes an ongoing project which seeks to contribute to a wider understanding of the realities of bridging the semantic gap in visual image retrieval. A comprehensive survey of the means by which real image retrieval transactions are realised is being undertaken. An image taxonomy has been developed, in order to provide a framework within which account may be taken of the plurality of image types, user needs and forms of textual metadata. Significant limitations exhibited by current automatic annotation techniques are discussed, and a possible way forward using ontologically supported automatic content annotation is briefly considered as a potential means of mitigating these limitations.


Image Retrieval Latent Semantic Analysis Salient Object Automatic Annotation Image Material 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12), 1349–1380 (2000)CrossRefGoogle Scholar
  2. 2.
    Zhao, R., Grosky, W.I.: Bridging the semantic gap in image retrieval. In: Shih, T.K. (ed.) Distributed multimedia databases: techniques & applications, pp. 14–36. Idea Group Publishing, Hershey (2002)Google Scholar
  3. 3.
    Jőrgensen, C.: Image retrieval: theory and research. The Scarecrow Press, Lanham (2003)Google Scholar
  4. 4.
    Enser, P.G.B.: Pictorial information retrieval (Progress in Documentation). Journal of Documentation 51(2), 126–170 (1995)CrossRefGoogle Scholar
  5. 5.
    Rasmussen, E.M.: Indexing images. In: Williams, M.E. (ed.) Annual Review of Information Science 32. Information Today (ASIS), Information Today, Medford, New Jersey, pp. 169–196 (1997)Google Scholar
  6. 6.
    Sandore, B. (ed.): Progress in visual information access and retrieval. Library Trends, 48(2), 283–524 (1999)Google Scholar
  7. 7.
    Shatford, S.: Analysing the subject of a picture; a theoretical approach. Cataloging & Classification Quarterly 6(3), 39–62 (1986)CrossRefGoogle Scholar
  8. 8.
    Barnard, K., Duygulu, P., Forsyth, D., De Freitas, N., Blei, D.M., Jordan, M.I.: Matching Words and Pictures. Journal of Machine Learning Research 3(6), 1107–1135Google Scholar
  9. 9.
    Jeon, J., Lavrenko, V., Manmatha, R.: Automatic image annotation and retrieval using cross-media relevance models. In: Proceedings of the 26th annual international ACM SIGIR conference on research and development in information retrieval, pp. 119–126. ACM Press, New York (2003), Google Scholar
  10. 10.
    Fan, J., Hangzai Luo, Y.G., Xu, G.: Automatic image annotation by using concept-sensitive salient objects for image content representation. In: Proceedings of the 27th annual international ACM SIGIR conference on research and development in information retrieval, pp. 361–368. ACM Press, New York (2004)Google Scholar
  11. 11.
    Lavrenko, V., Manmatha, R., Jeon, J.: A model for learning the semantics of pictures. In: Seventeenth Annual Conference on Neural Information Processing Systems (2003)Google Scholar
  12. 12.
    Zhao, R., Grosky, W.I.: From Features to Semantics: Some Preliminary Results. In: IEEE International Conference on Multimedia and Expo, New York (2000),
  13. 13.
    Monay, F., Gatica-Perez, D.: On image auto-annotation with latent space models. ACM Multimedia, 275–278 (2003)Google Scholar
  14. 14.
    Kosinov, S., Marchand-Maillet, S.: Hierarchical ensemble learning for multimedia categorisation and autoannotation. In: Proceedings IEEE Machine Learning for Signal Processing workshop (MLSP), Sao Luis, Brazil (2004)Google Scholar
  15. 15.
    Enser, P.G.B.: Query Analysis in a Visual Information Retrieval Context. Journal of Document and Text Management 1(1), 25–52 (1993)Google Scholar
  16. 16.
    Armitage, L.H., Enser, P.G.B.: Analysis of user need in image archives. Journal of Information Science 23(4), 287–299 (1997)CrossRefGoogle Scholar
  17. 17.
    Enser, P., Sandom, C.: Retrieval of Archival Moving Imagery - CBIR Outside the Frame? In: Lew, M., Sebe, N., Eakins, J.P. (eds.) CIVR 2002. LNCS, vol. 2383, pp. 202–214. Springer, Berlin (2002)CrossRefGoogle Scholar
  18. 18.
    Panofsky, E.: Meaning in the visual arts. Doubleday Anchor Books, Garden City (1955)Google Scholar
  19. 19.
    Cawkell, A.E.: Selected aspects of image processing and management: review and future prospects. Journal of Information Science 18(3), 179–192 (1992)CrossRefGoogle Scholar
  20. 20.
    Enser, P.: Visual image retrieval: seeking the alliance of concept-based and content-based paradigms. Journal of Information Science 26(4), 199–210 (2000)CrossRefGoogle Scholar
  21. 21.
    Edina: Education Image Gallery,
  22. 22.
    Wellcome Trust: Medical Photographic Library,
  23. 23.
    Science & Society Picture Library,
  24. 24.
    Corporation of London: Talisweb,
  25. 25.
    Town, C., Sinclair, D.: Language-based querying of image collections on the basis of an extensible ontology. Image and Vision Computing 22(3), 251–267 (2003)CrossRefGoogle Scholar
  26. 26.
    Jaimes, A., Smith, J.R.: Semi-automatic, Data-driven Construction of Multimedia Ontologies. In: Proceedings of the IEEE International Conference on Multimedia and Expo (2003),
  27. 27.
    Hollink, L., Schreiber, A., Wielemaker Th., J., Wielinga, B.: Semantic Annotation of Image Collections. In: Proceedings of the KCAP 2003 Workshop on Knowledge Capture and Semantic Annotation, Florida (2003),
  28. 28.
    Goodall, S., Lewis, P.H., Martinez, K., Sinclair, P.A.S., Giorgini, F., Addis, M.J., Laharnier, C., Stevenson, J.: Knowledge-based exploration of multimedia museum collections. In: Proceedings of the European workshop on the integration of knowledge semantics and digital media technology, London, pp. 415–422 (2004)Google Scholar
  29. 29.
    Addis, M., Boniface, M., Goodall, S., Grimwood, P., Kim, S., Lewis, P., Martinez, K., Stevenson, A.: SCULPTEUR: Towards a New Paradigm for Multimedia Museum Information Handling. In: Fensel, D., Sycara, K., Mylopoulos, J. (eds.) ISWC 2003. LNCS, vol. 2870, pp. 582–596. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  30. 30.
    Hu, B., Dasmahapatra, S., Lewis, P., Shadbolt, N.: Ontology-based Medical Image Annotation with Description Logics. In: Proceedings of the 15th IEEE International Conference on Tools with Artificial Intelligence, Sacramento, CA, USA (2003) (in press)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Peter G. B. Enser
    • 1
  • Christine J. Sandom
    • 1
  • Paul H. Lewis
    • 2
  1. 1.School of Computing, Mathematical and Information SciencesUniversity of Brighton 
  2. 2.Department of Electronics and Computer ScienceUniversity of Southampton 

Personalised recommendations