Skip to main content

Automatic image annotation approach based on optimization of classes scores

Computing volume 96, pages 381–402 (2014)Cite this article

Abstract

This article presents an automatic image level annotation approach that takes advantage of both context and semantics presented in segmented images. The proposed approach is based on the optimization of classes’ scores using particle swarm optimization. In addition, random forest classifier and normalized cuts algorithm have been applied for automatic image classification, annotation, and clustering. For the proposed approach, each input image is segmented using the normalized cuts segmentation algorithm in order to create a descriptor for each segment. Two parameter selection models have been selected for particle swarm optimization algorithm and many voting techniques have been implemented to find the most suitable set of annotation words per image. Experimental results, using Corel5k benchmark annotated images dataset, demonstrate that applying optimization algorithms along with random forest classifier achieved noticeable increase in image annotation performance measures compared to related researches on the same dataset.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

References

  1. Chen Z, Hou J, Zhang D, Qin X (2012) An annotation rule extraction algorithm for image retrieval. Pattern Recognit Lett 33(10):1257–1268

    Article  Google Scholar 

  2. Zhang D, Islam M, Lu G (2012) A review on automatic image annotation techniques. Pattern Recognit 45(1):346–362

    Article  Google Scholar 

  3. Wang Y, Mei T, Gong S, Hua X (2009) Combining global, regional and contextual features for automatic image annotation. Pattern Recognit 42(2):259–266

    Article  MATH  Google Scholar 

  4. Yao J, Zhang Z, Antani S, Long R, Thoma G (2008) Automatic medical image annotation and retrieval. Neurocomputing 71(10):2012–2022

    Article  Google Scholar 

  5. Yu N, Hua K, Cheng H (2012) A multi-directional search technique for image annotation propagation. J Vis Commun Image Represent 23(1):237–244

    Article  Google Scholar 

  6. Gao Y, YIN Y, Uozumi T (2012) A hierarchical image annotation method based on SVM and semi-supervised EM. Acta Automatica Sinica 36(7):960–967

    Article  Google Scholar 

  7. Qi X, Han Y (2007) Incorporating multiple SVM for automatic image annotation. Pattern Recognit 40(2):728–741

    Article  MATH  Google Scholar 

  8. Li R, Lu J, Zhang Y, Zhao T (2010) Dynamic adaboost learning with feature selection based on parallel genetic algorithm for image annotation. Knowl Based Syst 23(3):195–201

    Article  Google Scholar 

  9. Corel5k dataset website (online) (2012) http://kobus.ca/research/data/eccv_2002/. Accessed July 2012

  10. Hironobu YM, Takahashi H, Oka R (1999) Image-to-word transformation based on dividing and vector quantizing images with words. Boltzmann machines. Neural Netw 405–409

  11. Duygulu P, Barnard K, Freitas J, Forsyth D (2002) Object recognition as machine translation: learning a lexicon for a fixed image vocabulary. In: 7th European conference on computer vision (ECCV 2002). Springer, LNCS, Copenhagen, 28–31 May 2002, pp 97–112

  12. Jeon J, Lavrenko V, Manmatha R (2003) Automatic image annotation and retrieval using cross-media relevance models. In: 26th annual international ACM SIGIR conference on research and development in information retrieval. ACM, Toronto, 28 July–1 August 2003, pp 119–126

  13. Lavrenko V, Manmatha R, Jeon J (2003) A model for learning the semantics of pictures. In: 16th conference on advances in neural information processing systems (NIPS 16), Vancouver. MIT Press, Canada, 8–13 December 2003

  14. Feng SL, Manmatha R, Lavrenko V (2004) Multiple Bernoulli relevance models for image and video annotation. In: IEEE computer society conference on computer vision and pattern recognition (CVPR ’04). IEEE, Washington, 27 June–2 July 2004, pp 1002–1009

  15. Cusano C, Ciocca G, Schettini R (2004) Image annotation using SVM. Internet Imaging IV 5304(1):330–338

    Google Scholar 

  16. Zhu S, Tan X (2011) A novel automatic image annotation method based on multi-instance learning. Procedia Eng 15:3439–3444

    Article  Google Scholar 

  17. Wang L, Khan L (2006) Automatic image annotation and retrieval using weighted feature selection. Multimed Tools Appl 29(1):55–71

    Article  Google Scholar 

  18. Lu J, Zhao T, Zhang Y (2008) Feature selection based-on genetic algorithm for image annotation. Knowl Based Syst J 21(8):887–891

    Article  Google Scholar 

  19. Sun F, He JP (2009) A normalized cuts based image segmentation method. In: 2nd International conference on information and computing science. IEEE, Manchester, 21–22 May 2009, pp 333–336

  20. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905

    Article  Google Scholar 

  21. Clerc M, Kennedy J (2003) The particle swarm—explosion, stability, and convergence in a multidimensional complex space. IEEE Trans Evol Comput 6(1):58–73

    Article  Google Scholar 

  22. Trelea IC (2003) The particle swarm optimization algorithm: convergence analysis and parameter selection. Inf Process Lett 85(6):317–325

    Article  MATH  MathSciNet  Google Scholar 

  23. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: IEEE International conference on neural networks, vol. 4. IEEE, Perth, 27 November–1 December 1995, pp 1942–1948

  24. Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  MATH  Google Scholar 

  25. Ho TK (1995) Random decision forests. In: 3rd International conference on document analysis and recognition (ICDAR 1995), vol. 1. IEEE Computer Society, Montreal, 14–15 August 1995, pp 278–282

  26. Ho TK (1998) The random subspace method for constructing decision forests. IEEE Trans Pattern Anal Mach Intell 20(8):832–844

    Article  Google Scholar 

  27. Breiman L, Friedman JH, Olshen RA, Stone CJ (1984) Classification and regression trees. Chapman and Hall, New York

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Arab Academy for Science, Technology, and Maritime Transport, Cairo, Egypt

    Nashwa El-Bendary

  2. Scientific Research Group in Egypt (SRGE), Cairo, Egypt

    Nashwa El-Bendary, Aboul Ella Hassanien & Mohamed Sami

  3. Hannam University, Daejeon, Korea

    Tai-hoon Kim

  4. Faculty of Computers and Information, Cairo University, Cairo, Egypt

    Aboul Ella Hassanien & Mohamed Sami

Authors
  1. Nashwa El-Bendary
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tai-hoon Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aboul Ella Hassanien
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohamed Sami
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aboul Ella Hassanien.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

El-Bendary, N., Kim, Th., Hassanien, A.E. et al. Automatic image annotation approach based on optimization of classes scores. Computing 96, 381–402 (2014). https://doi.org/10.1007/s00607-013-0342-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00607-013-0342-0

Keywords

  • Image annotation
  • Segmentation
  • Particle swarm optimization
  • Random forest classifier

Mathematics Subject Classification

  • 68Uxx