Skip to main content
SpringerLink
Log in

Shape annotation for intelligent image retrieval

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Annotation of shapes is an important process for semantic image retrieval. In this paper, we present a shape annotation framework that enables intelligent image retrieval by exploiting in a unified manner domain knowledge and perceptual description of shapes. A semi-supervised fuzzy clustering process is used to derive domain knowledge in terms of linguistic concepts referring to the semantic categories of shapes. For each category we derive a prototype that is a visual template for the category. A novel visual ontology is proposed to provide a description of prototypes and their salient parts. To describe parts of prototypes the visual ontology includes perceptual attributes that are defined by mimicking the analogy mechanism adopted by humans to describe the appearance of objects. The effectiveness of the developed framework as a facility for intelligent image retrieval is shown through results on a case study in the domain of fish shapes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

Notes

  1. http://www.all-birds.com/Identify.htm

References

  1. Abbasi S, Mokhtarian F, Kittler J Squid demo dataset: http://www.ee.surrey.ac.uk/cvssp/demos/css/demo.html

  2. Al-Khatib W, Day YF, Ghafoor A, Berra PB (1999) Semantic modeling and knowledge representation in multimedia databases. IEEE Trans Knowl Data Eng 11(1):64–80

    Article  Google Scholar 

  3. Apache-Software-Foundation. http://incubator.apache.org/jena/index.html

  4. Bartolini I, Ciaccia P, Patella M (2005) WARP: accurate retrieval of shapes using phase of Fourier descriptors and Time warping distance. IEEE Trans Pattern Anal Mach Intell 27(1):142–147

    Article  Google Scholar 

  5. Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24(4):509–522

    Article  Google Scholar 

  6. Berners-Lee T, Hendler J, Lassila O (2001) The semantic web. Sci Am 284(5):28–37

    Article  Google Scholar 

  7. Bertini M, Bimbo AD, Serra G, Torniai C (2009) Dynamic pictorially enriched ontologies for digital video libraries. IEEE MultiMedia 2009:16

    Google Scholar 

  8. Bloehdorn S, Petridis K, Saathoff C, Simou N, Tzouvaras V, Avrithis Y, Handschuh S, Kompatsiaris Y, Staab S, Strintzis M (2005) Semantic annotation of images and videos for multimedia analysis. In: Gmez-Prez A, Euzenat J (eds) The semantic web: research and applications, volume 3532 of Lecture notes in computer science. Springer, Berlin/Heidelberg , pp 592–607

    Google Scholar 

  9. Borras A, Llados J (2005) Object image retrieval by shape content in complex scenes using geometric constraints. Pattern Recognit Image Anal. Springer Berlin/Heidelberg, pp 325–332

  10. Bouet M, Aufaure M-A (2007) New image retrieval principle: Image mining and visual ontology. In: Petrushin VA, Khan L (eds) Multi-media data mining and knowledge discovery. Springer, London, pp 168–184

    Chapter  Google Scholar 

  11. Carlin M (2001) Measuring the performance of shape similarity retrieval methods. Comput Vis Image Underst 84(1):44–61

    Article  MATH  Google Scholar 

  12. Castellano G, Fanelli A M, Torsello M A (2011) A fuzzy set approach for shape-based image annotation. In: Lecture notes on artificial intelligence, LNAI 6857, Springer-Verlag, pp 236–243

  13. Castellano G, Fanelli AM, Torsello MA (2011) Fuzzy image labeling by partially supervised shape clustering. In: Lecture notes on artificial intelligence, LNAI 6882, Springer-Verlag, pp 84–93

  14. Castellano G, Sforza G, Torsello M.A., Thinking of a system for image retrieval. In Proc. of the First Italian Information Retrieval Workshop (IIR2010) pp. 77–81 Padova, Italy (2010)

  15. Castellano G, Fanelli AM, Torsello MA (2014) Shape annotation by semi-supervised fuzzy clustering. Inf Sci 289:148–161

    Article  Google Scholar 

  16. Datta R, Dhiraj J, Jia L, Wang JZ (2008) Image retrieval: ideas, influences, and trends of the new age. ACM Comput Surv 40:1–60

    Article  Google Scholar 

  17. Eakins JP (2002) Towards intelligent image retrieval. Pattern Recogn 35(1):3–14

    Article  MATH  Google Scholar 

  18. Akbas E, Vural FY (2007) Automatic image annotation by ensemble of visual descriptors. In: Proceedings of conference on computer vision (CVPR) 2007, workshop on semantic learning applications in multimedia, pp 1–8

  19. Fan J, Gao Y, Luo H (2007) Hierarchical classification for automatic image annotation. In: Proceedings of the 30th annual international ACM SIGIR conference on research and development in information retrieval, pp 111–118

  20. Gardner H (1985) The minds new science: a history of the cognitive revolution. Basic Books, New York

    Google Scholar 

  21. Haralick R (1979) Statistical and structural approaches to texture. Proc IEEE 67(5):786–804

    Article  Google Scholar 

  22. Inoue M (2004) On the need for annotation-based image retrieval. In: Proceedings of the workshop on information retrieval in context, pp 44–46

  23. Jan J (2006) Medical image processing, reconstruction, and restoration: concepts and methods. Signal processing and communications. CRC Press

  24. Jiang SQ, Du J, Huang QM, Huang TJ, Gao W (2005) Visual ontology construction for digitized art image retrieval. J Comput Sci Technol 20(6):855–860

    Article  Google Scholar 

  25. Klassen E, Srivastava A, Mio W, Joshi SH (2004) Analysis of planar shapes using geodesic paths on shape spaces. IEEE Trans Pattern Anal Mach Intell 26(3):372–383

    Article  Google Scholar 

  26. Kompatsiaris I, Mezaris V, Strintzis MG (2005) Multimedia content indexing and retrieval using an object ontology. In: Multimedia content and the semantic web, pp 339–371

  27. Kontschieder P, Donoser M, Bischof H (2009) Beyond Pairwise Shape Similarity Analysis. In: Proceedings of asian conference on computer vision (ACCV), pp 655–666

  28. Li D, Simske S (2002) Shape retrieval based on distance ratio distribution. HP Tech Report. HPL-2002-251

  29. Liang X, Zhuang Q, Cao N, Zhang J (2009) Shape modeling and clustering of white matter fiber tracts using Fourier descriptors Proceedings of the 6th annual IEEE conference on computational intelligence in bioinformatics and computational biology, pp 292–297

  30. Liu Y, Zhang D, Lu G, Ma W-Y (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recogn 40(1):262–282

    Article  MATH  Google Scholar 

  31. Liu Y, Zhang J, Tjondronegoro D, Geve S (2007) A shape ontology framework for bird classification. In: Proceedings of 9th biennial conference of the IEEE australian pattern recognition society, pp 478–484

  32. Liu Y, Zhang J, Tjondronegoro D, Geve S, Li Z (2010) Mid-level concept learning with visual contextual ontologies and probabilistic inference for image annotation. In: Boll S, Tian Q, Zhang L, Zhang Z, Chen Y-P (eds) Advances in multimedia modeling, volume 5916 of Lecture notes in computer science. Springer, Berlin/Heidelberg, pp 229–239

    Google Scholar 

  33. Maillot NE, Thonnat M (2008) Ontology based complex object recog- nition. Image Vis Comput 26 (1):102–113

    Article  Google Scholar 

  34. Muda Z (2007) Classification and image annotation for bridging the semantic gap. In: Summer school on multimedia semantics. University of Glasgow, Glasgow, p 2007

  35. Muller H, Michoux N, Bandon D, Geissbuhler A (2004) A review of content-based image retrieval systems in medical applications–clinical benefits and future directions. Int J Med Inform 73(1):1–23

    Article  Google Scholar 

  36. Niblack W, Barber R, Equitz W, Flickner M, Glasman E, Petkovic D, Yanker P (1993) The Qbic project: querying images by content using color, texture, and shape. In: Proceedings of SPIE conference on storage and retrieval of image and video databases , pp 1–8

  37. Noy NF, Sintek M, Decker S, Crubezy M, Fergerson RW, Musen MA (2001) Creating semantic web contents with protege. IEEE Intell Syst 16(2):60–71

    Article  Google Scholar 

  38. Pedrycz W, Waletzky J (1997) Fuzzy clustering with partial supervision. IEEE Trans Syst Man Cybern 27(5):787–795

    Article  Google Scholar 

  39. Plataniotis KN, Venetsanopoulos AN (2000) Color image processing and applications. Springer, Berlin

    Book  Google Scholar 

  40. Rajpoot NM, Arif M (2008) Unsupervised shape clustering using diffusion maps. Ann BMVA 2008(5):1–17

    Google Scholar 

  41. Rui Y, Huang T, Chang S (1999) Image retrieval: current techniques, promising directions and open issues. J Vis Commun Image Represent 10(4):39–62

    Article  Google Scholar 

  42. Russell BC, Torralba A, Murphy KP, Freeman WT (2008) LabelMe: a database and web-based tool for image annotation. Int J Comput Vis 77(1–3):157–173

    Article  Google Scholar 

  43. Simou N, Tzouvaras V, Avrithis Y, Stamou G, Kollias S (2005) A visual descriptor ontology for multimedia reasoning. In: Proceedings of workshop on image analysis for multimedia interactive services (WIAMIS2005). Montreux, Switzerland , pp 13–15

  44. Smeulders AWM, Worring M, Santini S, Gupta A, Jain R (2000) Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22:1349–1380

    Article  Google Scholar 

  45. Stamou G, Kollias S (2005) Multimedia content and the semantic web: standards, methods and tools. Wiley

  46. Sforza G (2012) “Image content representation: using low-level and high-level descriptions”, PhD thesis, Universita’ degli Studi di Bari Aldo Moro, Italy

  47. Uren V, Cimiano P, Iria J, Handschuh S, Vargas-Vera M, Motta E, Ciravegna F (2006) Semantic annotation for knowledge management: requirements and a survey of the state of the art. Web Semant Sci Serv Agents World Wide Web 4(1): 14–18

    Article  Google Scholar 

  48. Veltkamp R, Tanase M (2001) Content-based image retrieval systems: a survey. Technical Report

  49. Yang M, Kpalma K, Ronsin J (2008) A survey of shape feature ex- traction techniques. Pattern Recogn:43–90

  50. Zhang D, Islam MM, Lu G (2011) A review on automatic image annotation techniques. Pattern Recogn 45(1):346–362

    Article  Google Scholar 

Download references

Acknowledgments

Funding for this work was provided by the Fondazione Cassa di Risparmio di Puglia (FCRP), that supported the Italian project “Annotazione di forme per la ricerca intelligente di immagini”.

Author information

Authors and Affiliations

  1. Department of Informatics, University of Bari “A. Moro”, Via Orabona, 4 - 70126, Bari, Italy

    Giovanna Castellano, Anna M. Fanelli & M. Alessandra Torsello

  2. Department of Computer Science, University of Milano, Via Bramante, 65 - 26013, Crema, Italy

    Gianluca Sforza

Authors
  1. Giovanna Castellano
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anna M. Fanelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gianluca Sforza
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. M. Alessandra Torsello
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Giovanna Castellano.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Castellano, G., Fanelli, A.M., Sforza, G. et al. Shape annotation for intelligent image retrieval. Appl Intell 44, 179–195 (2016). https://doi.org/10.1007/s10489-015-0693-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-015-0693-7

Keywords

Search

Navigation