Cognitive Processing

, Volume 10, Issue 3, pp 233–242

Improving image annotation via useful representative feature selection

  • Wei-Chao Lin
  • Michael Oakes
  • John Tait
  • Chih-Fong Tsai
Research Report


This paper describes the automatic assignment of images into classes described by individual keywords provided with the Corel data set. Automatic image annotation technology aims to provide an efficient and effective searching environment for users to query their images more easily, but current image retrieval systems are still not very accurate when assigning images into a large number of keyword classes. Noisy features are the main problem, causing some keywords never to be assigned to their correct images. This paper focuses on improving image classification, first by selection of features to characterise each image, and then the selection of the most suitable feature vectors as training data. A Pixel Density filter (PDfilter) and Information Gain (IG) are proposed to perform these respective tasks. We filter out the noisy features so that groups of images can be represented by their most important values. The experiments use hue, saturation and value (HSV) colour feature space to categorise images according to one of 190 concrete keywords or subsets of these. The study shows that feature selection through the PDfilter and IG can improve the problem of spurious similarity.


Image annotation Image retrieval Information gain 


  1. Baeza-Yates R, Ribeiro-Neto B (1999) Modern information retrieval. Addison Wesley, EnglandGoogle Scholar
  2. Barnard K, Duygulu P, Forsyth D, de Freitas N, Blei DM, Jordan MI (2003) Matching words and pictures. J Mach Learn Res 3:1107–1135CrossRefGoogle Scholar
  3. Belew RK (2000) Finding out about: a cognitive perspective on search engine technology and the WWW. Cambridge University Press, CambridgeGoogle Scholar
  4. Bishop CM (2006) Pattern recognition and machine learning. Springer, New YorkGoogle Scholar
  5. Blei DM, Ng AY, Jordan MI (2003) Latent Dirichlet allocation. J Mach Learn Res 3:993–1022CrossRefGoogle Scholar
  6. Breiman L, Friedman JH, Olshen RA, Stone CJ (1984) Classification and regression trees. CRC Press, Boca RatonGoogle Scholar
  7. Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Mining Knowl Discov 2(2):121–167CrossRefGoogle Scholar
  8. Carneiro G, Chan AB, Moreno PJ, Vasconcelos N (2007) Supervised learning of semantic classes for image annotation and retrieval. IEEE Trans Pattern Anal Mach Intell 29(3):394–410PubMedCrossRefGoogle Scholar
  9. Daubechies I (1992) Ten lectures on wavelets. Society for Industrial and Applied Mathematics, PhiladelphiaGoogle Scholar
  10. Del Bimbo A (1996) Image and video databases: visual browsing, querying and retrieval. J Vis Lang Comput 7(4):353–359CrossRefGoogle Scholar
  11. Del Bimbo A (1999) Visual information retrieval. Morgan Kaufmann, San FranciscoGoogle Scholar
  12. Eakins JP, Graham ME (1999) Content-based image retrieval: a report of the JISC technology applications programme. The Joint Information Systems Committee (JISC). (26 January 2007)
  13. Gonzalez RC, Woods RE, Eddins SL (2004) Digital image processing using MATLAB. Pearson Prentice-Hall, Upper Saddle RiverGoogle Scholar
  14. Grubinger M, Clough P, Müller H, Deselaers T (2006) The IAPR TC-12 Benchmark—a new evaluation resource for visual information systems. In: Proceedings of the International Workshop OntoImage’2006 Language Resources for Content-Based Image Retrieval, held in conjunction with LREC’06. Genoa, Italy, 22 May 2006, pp 13–23Google Scholar
  15. Gupta A, Santini S, Jain R (1997) In search of information in visual media. Commun ACM 40(12):35–42CrossRefGoogle Scholar
  16. Howarth P, Rüger S (2004) Evaluation of texture features for content-based image retrieval. International Conference on Image and Video Retrieval (CIVR), Dublin, pp 326–334Google Scholar
  17. Idris F, Panchanathan S (1997) Review of image and video indexing techniques. J Vis Commun Image Represent 8(2):146–166CrossRefGoogle Scholar
  18. Jain AK, Duin RPW, Mao J (2000) Statistical pattern recognition: a review. IEEE Trans Pattern Anal Mach Intell 22(1):4–37CrossRefGoogle Scholar
  19. Jeon J, Manmatha R (2004) using maximum entropy for automatic image annotation. In: Proceedings of the International Conference on Image and Video Retrieval, Dublin, Ireland, July 21–23 2004: 24–32Google Scholar
  20. Jin X, French JC (2003) Improving image retrieval effectiveness via multiple queries. In: Proceedings of the First ACM International Workshop on Multimedia Database, New Orleans, LA, USA, pp 86–93Google Scholar
  21. Jörgensen C, Jaimes A, Benitez AB, Chang S (2001) A conceptual framework and research for classifying visual descriptors. J Am Soc Inf Sci 52(11):938–947 Special Issue on Image Access: Bridging Multiple Needs and Multiple PerspectivesCrossRefGoogle Scholar
  22. Lai T (2000) CHROMA: a photographic image retrieval system. PhD Thesis. University of Sunderland, UKGoogle Scholar
  23. Lavrenko V, Manmatha R, Jeon J (2003) A model for learning the semantics of pictures. In: Proceedings of the International Conference on Neural Information Processing Systems, Vancouver, Canada, 8–13 December 2003Google Scholar
  24. Lew MS (2001) Principles of visual information retrieval. Springer, LondonGoogle Scholar
  25. Li J, Wang JZ (2003) Automatic linguistic indexing of pictures by a statistical modelling approach. IEEE Trans Pattern Anal Mach Intell 25(9):1075–1088CrossRefGoogle Scholar
  26. Long F, Zhang H, Feng DD (2003) Fundamentals of content-based image retrieval. In: Feng DD, Siu WC, Zhang H (eds) Multimedia information retrieval and management—technological fundamentals and applications. Springer, GermanyGoogle Scholar
  27. Mandal MK, Idris F, Panchanathan S (1999) A critical evaluation of image and video indexing techniques in the compressed domain. Image Vis Comput 17(7):513–529CrossRefGoogle Scholar
  28. Manning CD, Schütze H (1999) Foundations of statistical natural language processing. MIT, LondonGoogle Scholar
  29. Mathias E, Conci A (1998) Comparing the influence of color spaces and metrics in content based image retrieval. In: Proceedings of the IEEE International Symposium on Computer Graphics, Image Processing, and Vision. Rio de Janeiro, Brazil, 20–23 October 1998, pp 371–378Google Scholar
  30. Mitchell TM (1997) Machine learning. McGraw Hill, New YorkGoogle Scholar
  31. Müller H, Müller W, Marchand-Maillet S, Pun T, Squire DM (2003) A framework for benchmarking in CBIR. Multimedia Tools Appl 21(1):55–73CrossRefGoogle Scholar
  32. Oakes MP (1998) Statistics for corpus linguistics. Edinburgh University Press, EdinburghGoogle Scholar
  33. Quinlan JR (1986) Induction of decision trees. Mach Learn 1(1):81–106Google Scholar
  34. Shanbehzadeh J, Moghadam AME, Mahmoudi F (2000) Image indexing and retrieval techniques: past, present, and next. In: Proceedings of SPIE, The International Society for Optical Engineering, 3972, pp 461–490Google Scholar
  35. Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J. 27, July and October: 379–423 and 623–656Google Scholar
  36. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905CrossRefGoogle Scholar
  37. Smeaton AF, Kraaij W, OverP (2004) The TREC video retrieval evaluation (TRECVID): a case study and status report. In: Proceedings of the RIAO 2004 Conference. Avignon, France, 26–28 April 2004, pp 25–37Google Scholar
  38. Swain MJ, Ballard DH (1991) Color indexing. Int J Comp Vis 7(1):11–32CrossRefGoogle Scholar
  39. Tsai C (2005) Automatically annotating images with keywords. PhD Thesis, University of Sunderland, UKGoogle Scholar
  40. Tsai C, McGarry K, Tait J (2006) Qualitative evaluation of automatic assignment of keywords to images. Inf Process Manage 42(1):136–154CrossRefGoogle Scholar
  41. Vailaya A (2000) Semantic classification in image databases. PhD Thesis. Michigan State University, USAGoogle Scholar
  42. van der Heijden F (1994) Image based measurement systems: object recognition and parameter estimation. Wiley, ChichesterGoogle Scholar
  43. Wu JK, Kankanhalli MS, Lim J, Hong D (2000) Perspectives on content-based multimedia systems. Kluwer Academic Publishers, LondonGoogle Scholar
  44. Yang Y, Pedersen JO (1997) A comparative study on feature selection in text categorization. In: Proceedings of the Fourteenth International Conference on Machine Learning. 8–12 July 1997, pp 412–420Google Scholar

Copyright information

© Marta Olivetti Belardinelli and Springer-Verlag 2008

Authors and Affiliations

  • Wei-Chao Lin
    • 1
  • Michael Oakes
    • 1
  • John Tait
    • 2
  • Chih-Fong Tsai
    • 3
  1. 1.Department of Computing, Engineering and TechnologyUniversity of SunderlandSunderlandUK
  2. 2.Information Retrieval FacilityViennaAustria
  3. 3.Department of Information ManagementNational Central UniversityJhongliTaiwan

Personalised recommendations