Skip to main content
SpringerLink
Log in

Adding semantics to image-region annotations with the Name-It-Game

  • Interactive Multimedia Computing
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

In this paper we present the Name-It-Game, an interactive multimedia game fostering the swift creation of a large data set of region-based image annotations. Compared to existing annotation games, we consider an added semantic structure, by means of the WordNet ontology, the main innovation of the Name-It-Game. Using an ontology-powered game, instead of the more traditional annotation tools, potentially makes region-based image labeling more fun and accessible for every type of user. However, the current games often present the players with hard-to-guess objects. To prevent this from happening in the Name-It-Game, we successfully identify WordNet categories which filter out hard-to-guess objects. To verify the speed of the annotation process, we compare the online Name-It-Game with a desktop tool with similar features. Results show that the Name-It-Game outperforms this tool for semantic region-based image labeling. Lastly, we measure the accuracy of the produced segmentations and compare them with carefully created LabelMe segmentations. Judging from the quantitative and qualitative results, we believe the segmentations are competitive to those of LabelMe, especially when averaged over multiple games. By adding semantics to region-based image annotations, using the Name-It-Game, we have opened up an efficient means to provide precious labels in a playful manner.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. http://hunch.net/?p=170.

References

  1. Hanbury, A.: A survey of methods for image annotation. J. Vis. Lang. Comput. 19(5), 617–627 (2008)

    Article  Google Scholar 

  2. Datta, R., Joshi, D., Li, J., Wang, J.Z.: Image retrieval: ideas, influences, and trends of the new age. ACM Comput. Surv. 40(65), 1–60 (2008)

    Article  Google Scholar 

  3. Enser, P.: Visual image retrieval: seeking the alliance of concept-based and content-based paradigms. J. Inf. Sci. 26(4), 199–210 (2000)

    Article  Google Scholar 

  4. Golder, S.A., Huberman, B.A.: The structure of collaborative tagging systems. J. Inf. Sci. 32(2), 198–208 (2006)

    Article  Google Scholar 

  5. Macgregor, G., McCulloch, E.: Collaborative tagging as a knowledge organisation and resource discovery tool. Libr. Rev. 55(5), 291–300 (2006)

    Article  Google Scholar 

  6. Russell, B.C., Torralba, A., Murphy, K.P., Freeman, W.T.: LabelMe: a database and web-based tool for image annotation. Int. J. Comput. Vis. 77(1–3), (2008)

  7. Barnard, K., Fan, Q., Swaminathan, R., Hoogs, A., Collins, R., Rondot, P., Kaufhold, J.: Evaluation of localized semantics: data, methodology, and experiments. Int. J. Comput. Vis. 77(1–3), 199–217 (2008)

    Article  Google Scholar 

  8. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The Pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)

    Article  Google Scholar 

  9. Brostow, G.J., Fauqueur, J., Cipolla, R.: Semantic object classes in video: a high-definition ground truth database. Pattern Recognit. Lett. 30(2), 88–97 (2009)

    Article  Google Scholar 

  10. Naphade, M.R., Smith, J.R., Tešić, J., Chang, S.-F., Hsu, W., Kennedy, L.S., Hauptmann, A.G., Curtis, J.: Large-scale concept ontology for multimedia. IEEE MultiMed. 13(3), 86–91 (2006)

    Article  Google Scholar 

  11. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: A large-scale hierarchical image database. In: Proceedings IEEE Computer Vision and Pattern Recognition (2009)

  12. Chua, T.-S., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y.-T.: NUS-WIDE: A real-world web image database from National University of Singapore. In: Proceedings ACM International Conference on Image and Video Retrieval (2009)

  13. Fellbaum, C. (ed): WordNet: an electronic lexical database. The MIT Press, Cambridge, USA (1998)

    MATH  Google Scholar 

  14. Hollink, L., Schreiber, G., Wielemaker, J., Wielinga, B.: Semantic annotation of image collections. In: Proceedings international conference for Knowledge Capture Workshop on Knowledge Markup and Semantic Annotation (2003)

  15. Hyvönen, E., Styrman, A., Saarela, S.: Ontology-based image retrieval. In: Proceedings XML Finland conference, pp. 15–27 (2002)

  16. Gao, Y., Fan, J.: Incorporating concept ontology to enable probabilistic concept reasoning for multi-level image annotation. In: Proceedings ACM International Workshop on Multimedia Information Retrieval, pp. 79–88 (2006)

  17. von Ahn, L., Dabbish, L.: Labeling images with a computer game. In: Proceedings SIGCHI Conference on Human Factors in Computing Systems, pp. 319–326 (2004)

  18. von Ahn, L., Liu, R., Blum, M.: Peekaboom: a game for locating objects in images. In: Proceedings SIGCHI conference on Human Factors in Computing Systems, pp. 55–64 (2006)

  19. Squigl, http://www.gwap.com

  20. Turnbull, D., Liu, R., Barrington, L., Lanckriet, G.: A game-based approach for collecting semantic annotations of music. In: Proceedings International Conference on Music Information Retrieval (2007)

  21. Law, E.L.M., von Ahn, L., Dannenberg, R.B., Crawford, M.: TagATune: A game for music and sound annotation. In: Proceedings International Conference on Music Information Retrieval (2007)

  22. van Zwol, R., Garcia, L., Ramirez, G., Sigurbjörnsson, B., Labad, M.: Video tag game. In: Proceedings International World Wide Web Conference (2008)

  23. Gligorov, R., Baltussen, L.B., van Ossenbruggen, J., Aroyo, L., Brinkerink, M., Oomen, J., van Ees, A.: Towards integration of end-user tags with professional annotations. In: Proceedings International Web Science Conference (2010)

  24. Gonçalves, D., Jesus, R., Grangeiro, F., Romao, T., Correia, N.: Tag around: a 3D gesture game for image annotation. In: Proceedings International Conference on Advances in Computer Entertainment Technology, pp. 259–262 (2008)

  25. Seneviratne, L., Izquierdo, E.: An interactive framework for image annotation through gaming. In: Proceedings ACM International Conference on Multimedia Information Retrieval, pp. 517–526 (2010)

  26. Yao, B., Yang, X., Zhu, S.-C.: Introduction to a large-scale general purpose ground truth database: Methodology, annotation tool and benchmarks. In: Energy Minimization Methods in Computer Vision and Pattern Recognition, vol. 4679, LNCS, pp. 169–183, Springer (2007)

  27. Petridis, K., Anastasopoulos, D., Saathoff, C., Timmermann, N., Kompatsiaris, Y., Staab, S.: M-ontomat-annotizer: Image annotation linking ontologies and multimedia low-level features. In: B. Gabrys, R.J. Howlett, and L.C. Jain, editors, KES (3), vol. 4253 of LNCS, pp. 633–640, Springer (2006)

  28. Halaschek-Wiener, C., Golbeck, J., Schain, A., Grove, M., Parsia, B., Hendler, J.: Photostuff—an image annotation tool for the semantic web. In: Proceedings International Semantic Web Conference (2005)

  29. Hollink, L., Nguyen, G., Schreiber, G., Wielemaker, J., Wielinga, B., Worring, M.: Adding spatial semantics to image annotations. In: Proceedings International Workshop on Knowledge Markup and Semantic Annotation at ISWC (2004)

  30. Flickr, http://www.flickr.com

  31. Volkmer, T., Smith, J.R., Natsev, A.(P.): A web-based system for collaborative annotation of large image and video collections: an evaluation and user study. In: Proceedings ACM international conference on Multimedia, pp. 892–901 (2005)

  32. Marlow, C., Naaman, M., Boyd, D., Davis, M.: Ht06, tagging paper, taxonomy, flickr, academic article, to read. In: Proceedings International Conference on Hypertext and Hypermedia, pp. 31–40 (2006)

  33. Ames, M., Naaman, M.: Why we tag: Motivations for annotation in mobile and online media. In: Proceedings SIGCHI Conference on Human Factors in Computing Systems, pp. 971–980 (2007)

  34. Sorokin, A., Forsyth, D.: Utility data annotation with amazon mechanical turk. In: Proceedings IEEE Computer Vision and Pattern Recognition Workshops (2008)

Download references

Acknowledgments

This work is supported by the STW SEARCHER project.

Author information

Authors and Affiliations

  1. University of Amsterdam, Science Park 904, 1098 XH, Amsterdam, The Netherlands

    Jeroen Steggink & Cees G. M. Snoek

Authors
  1. Jeroen Steggink
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cees G. M. Snoek
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cees G. M. Snoek.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Steggink, J., Snoek, C.G.M. Adding semantics to image-region annotations with the Name-It-Game. Multimedia Systems 17, 367–378 (2011). https://doi.org/10.1007/s00530-010-0220-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-010-0220-y

Keywords

Search

Navigation