Multimedia Tools and Applications

, Volume 70, Issue 1, pp 475–493 | Cite as

From global image annotation to interactive object segmentation

  • Xavier Giró-i-NietoEmail author
  • Manuel Martos
  • Eva Mohedano
  • Jordi Pont-Tuset


This paper presents a graphical environment for the annotation of still images that works both at the global and local scales. At the global scale, each image can be tagged with positive, negative and neutral labels referred to a semantic class from an ontology. These annotations can be used to train and evaluate an image classifier. A finer annotation at a local scale is also available for interactive segmentation of objects. This process is formulated as a selection of regions from a precomputed hierarchical partition called Binary Partition Tree. Three different semi-supervised methods have been presented and evaluated: bounding boxes, scribbles and hierarchical navigation. The implemented Java source code is published under a free software license.


Interaction Segmentation Multiscale Annotation Hierarchical 


  1. 1.
    Arbeláez P, Maire M, Fowlkes CC, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916. doi: 10.1109/TPAMI.2010.161 CrossRefGoogle Scholar
  2. 2.
    Bishop CM (2006) Pattern recognition and machine learning. SpringerGoogle Scholar
  3. 3.
    Carcel E, Martos M, Giro-i Nieto X, Marques F (2011) Rich internet applications for semi-automatic annotation of semantic shots in keyframes. In: MUSCLE Intl. Workshop. PisaGoogle Scholar
  4. 4.
    Cardoso J (2007) The semantic web vision: Where are we? IEEE Intell Syst 22(5):84–88. doi: 10.1109/MIS.2007.4338499 CrossRefGoogle Scholar
  5. 5.
    Chang CC, Lin CJ (2011) LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2:27:1–27:27. Software available at CrossRefGoogle Scholar
  6. 6.
    Dasiopoulou S, Giannakidou E, Litos G, Malasioti P, Kompatsiaris Y (2011) A survey of semantic image and video annotation tools. In: Paliouras G, Spyropoulos C, Tsatsaronis G (eds) Knowledge-driven multimedia information extraction and ontology evolution, Lecture notes in computer science, vol 6050. Springer, Berlin / Heidelberg, pp 196–239CrossRefGoogle Scholar
  7. 7.
    Fellbaum C (2010) Wordnet. In: Poli R, Healy M, Kameas A (eds) Theory and applications of ontology: computer applications. Springer, Netherlands, pp 231–243CrossRefGoogle Scholar
  8. 8.
    Giro-i Nieto X, Camps N, Marques F (2010) Gat, a graphical annotation tool for semantic regions. Multimed Tools Appl 46(2):155–174. doi: 10.1007/s11042-009-0389-2 CrossRefGoogle Scholar
  9. 9.
    Giro-i Nieto X, Ventura C, Pont-Tuset J, Cortes S, Marques F (2010) System architecture of a web service for content-based image retrieval. In: ACM intl. conference on image and video retrieval, CIVR ’10, pp 358–365. doi: 10.1145/1816041.1816093
  10. 10.
    Gulshan V, Rother C, Criminisi A, Blake A, Zisserman A (2010) Geodesic star convexity for interactive image segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognitionGoogle Scholar
  11. 11.
    Hanbury A (2008) A survey of methods for image annotation. J Visual Lang Comput 19(5):617–627. doi: 10.1016/j.jvlc.2008.01.002 CrossRefGoogle Scholar
  12. 12.
    Lempitsky VS, Kohli P, Rother C, Sharp T (2009) Image segmentation with a bounding box prior. In: IEEE international conference on computer vision, pp 277–284. doi: 10.1109/ICCV.2009.5459262
  13. 13.
    Manjunath BS, Salembier P, Sikora T (eds) (2002) Introduction to MPEG-7: Multimedia content description interface. Wiley, Chichester, West Sussex, UKGoogle Scholar
  14. 14.
    McGuinness K, O’Connor NE (2010) A comparative evaluation of interactive segmentation algorithms. Pattern Recogn 43(2):434–444. doi: 10.1016/j.patcog.2009.03.008 CrossRefzbMATHGoogle Scholar
  15. 15.
    Mezaris V, Kompatsiaris I, Strintzis MG (2004) Region-based image retrieval using an object ontology and relevance feedback. EURASIP J Appl Signal Process 2004:886–901. doi: 10.1155/S1110865704401188 CrossRefGoogle Scholar
  16. 16.
    Naphade M, Smith J, Tesic J, Chang SF, Hsu W, Kennedy L, Hauptmann A, Curtis J (2006) Large-scale concept ontology for multimedia. IEEE Multimed 13(3):86–91. doi: 10.1109/MMUL.2006.63 CrossRefGoogle Scholar
  17. 17.
    Noma A, Graciano AB, Cesar RM, Consularo LA, Bloch I (2012) Interactive image segmentation by matching attributed relational graphs. Pattern Recogn 45(3):1159–1179. doi: 10.1016/j.patcog.2011.08.017 CrossRefGoogle Scholar
  18. 18.
    Petridis K, Anastasopoulos D, Saathoff C, Kompatsiaris Y, Staab S (2006) Montomat-annotizer: image annotation, linking ontologies and multimedia low-level features. In: Intl. conf. on knowledge based, intelligent information and engineering systemsGoogle Scholar
  19. 19.
    Rother C, Kolmogorov V, Blake A (2004) “grabcut”: interactive foreground extraction using iterated graph cuts. ACM Trans Graph 23(3):309–314. doi: 10.1145/1015706.1015720 CrossRefGoogle Scholar
  20. 20.
    Russell BC, Torralba A, Murphy KP, Freeman WT (2008) Labelme: A database and web-based tool for image annotation. Int J Comput Vis 77(1–3):157–173. doi: 10.1007/s11263-007-0090-8 CrossRefGoogle Scholar
  21. 21.
    Salembier P, Garrido L (2000) Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval. IEEE Trans Image Process 9(4):561–576. doi: 10.1109/83.841934 CrossRefGoogle Scholar
  22. 22.
    Steggink J, Snoek C (2011) Adding semantics to image-region annotations with the name-it-game. Multimedia Syst 17:367–378. doi: 10.1007/s00530-010-0220-y CrossRefGoogle Scholar
  23. 23.
    Volkmer T, Smith JR, Natsev AP (2005) A web-based system for collaborative annotation of large image and video collections: an evaluation and user study. In: ACM intl. conference on multimedia, pp 892–901. doi: 10.1145/1101149.1101341
  24. 24.
    von Ahn L, Dabbish L (2004) Labeling images with a computer game. In: SIGCHI conference on Human factors in computing systems, pp 319–326. doi: 10.1145/985692.985733

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Xavier Giró-i-Nieto
    • 1
    Email author
  • Manuel Martos
    • 1
  • Eva Mohedano
    • 1
  • Jordi Pont-Tuset
    • 1
  1. 1.Campus Nord UPC (Mòdul D5)BarcelonaSpain

Personalised recommendations