Multimedia Systems

, Volume 17, Issue 5, pp 367–378 | Cite as

Adding semantics to image-region annotations with the Name-It-Game

Interactive Multimedia Computing
First Online:

Abstract

In this paper we present the Name-It-Game, an interactive multimedia game fostering the swift creation of a large data set of region-based image annotations. Compared to existing annotation games, we consider an added semantic structure, by means of the WordNet ontology, the main innovation of the Name-It-Game. Using an ontology-powered game, instead of the more traditional annotation tools, potentially makes region-based image labeling more fun and accessible for every type of user. However, the current games often present the players with hard-to-guess objects. To prevent this from happening in the Name-It-Game, we successfully identify WordNet categories which filter out hard-to-guess objects. To verify the speed of the annotation process, we compare the online Name-It-Game with a desktop tool with similar features. Results show that the Name-It-Game outperforms this tool for semantic region-based image labeling. Lastly, we measure the accuracy of the produced segmentations and compare them with carefully created LabelMe segmentations. Judging from the quantitative and qualitative results, we believe the segmentations are competitive to those of LabelMe, especially when averaged over multiple games. By adding semantics to region-based image annotations, using the Name-It-Game, we have opened up an efficient means to provide precious labels in a playful manner.

Keywords

Image-region annotation Ontology Labeling game 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

This work is supported by the STW SEARCHER project.

References

  1. 1.
    Hanbury, A.: A survey of methods for image annotation. J. Vis. Lang. Comput. 19(5), 617–627 (2008)CrossRefGoogle Scholar
  2. 2.
    Datta, R., Joshi, D., Li, J., Wang, J.Z.: Image retrieval: ideas, influences, and trends of the new age. ACM Comput. Surv. 40(65), 1–60 (2008)CrossRefGoogle Scholar
  3. 3.
    Enser, P.: Visual image retrieval: seeking the alliance of concept-based and content-based paradigms. J. Inf. Sci. 26(4), 199–210 (2000)CrossRefGoogle Scholar
  4. 4.
    Golder, S.A., Huberman, B.A.: The structure of collaborative tagging systems. J. Inf. Sci. 32(2), 198–208 (2006)CrossRefGoogle Scholar
  5. 5.
    Macgregor, G., McCulloch, E.: Collaborative tagging as a knowledge organisation and resource discovery tool. Libr. Rev. 55(5), 291–300 (2006)CrossRefGoogle Scholar
  6. 6.
    Russell, B.C., Torralba, A., Murphy, K.P., Freeman, W.T.: LabelMe: a database and web-based tool for image annotation. Int. J. Comput. Vis. 77(1–3), (2008)Google Scholar
  7. 7.
    Barnard, K., Fan, Q., Swaminathan, R., Hoogs, A., Collins, R., Rondot, P., Kaufhold, J.: Evaluation of localized semantics: data, methodology, and experiments. Int. J. Comput. Vis. 77(1–3), 199–217 (2008)CrossRefGoogle Scholar
  8. 8.
    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The Pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)CrossRefGoogle Scholar
  9. 9.
    Brostow, G.J., Fauqueur, J., Cipolla, R.: Semantic object classes in video: a high-definition ground truth database. Pattern Recognit. Lett. 30(2), 88–97 (2009)CrossRefGoogle Scholar
  10. 10.
    Naphade, M.R., Smith, J.R., Tešić, J., Chang, S.-F., Hsu, W., Kennedy, L.S., Hauptmann, A.G., Curtis, J.: Large-scale concept ontology for multimedia. IEEE MultiMed. 13(3), 86–91 (2006)CrossRefGoogle Scholar
  11. 11.
    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: A large-scale hierarchical image database. In: Proceedings IEEE Computer Vision and Pattern Recognition (2009)Google Scholar
  12. 12.
    Chua, T.-S., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y.-T.: NUS-WIDE: A real-world web image database from National University of Singapore. In: Proceedings ACM International Conference on Image and Video Retrieval (2009)Google Scholar
  13. 13.
    Fellbaum, C. (ed): WordNet: an electronic lexical database. The MIT Press, Cambridge, USA (1998)MATHGoogle Scholar
  14. 14.
    Hollink, L., Schreiber, G., Wielemaker, J., Wielinga, B.: Semantic annotation of image collections. In: Proceedings international conference for Knowledge Capture Workshop on Knowledge Markup and Semantic Annotation (2003)Google Scholar
  15. 15.
    Hyvönen, E., Styrman, A., Saarela, S.: Ontology-based image retrieval. In: Proceedings XML Finland conference, pp. 15–27 (2002)Google Scholar
  16. 16.
    Gao, Y., Fan, J.: Incorporating concept ontology to enable probabilistic concept reasoning for multi-level image annotation. In: Proceedings ACM International Workshop on Multimedia Information Retrieval, pp. 79–88 (2006)Google Scholar
  17. 17.
    von Ahn, L., Dabbish, L.: Labeling images with a computer game. In: Proceedings SIGCHI Conference on Human Factors in Computing Systems, pp. 319–326 (2004)Google Scholar
  18. 18.
    von Ahn, L., Liu, R., Blum, M.: Peekaboom: a game for locating objects in images. In: Proceedings SIGCHI conference on Human Factors in Computing Systems, pp. 55–64 (2006)Google Scholar
  19. 19.
  20. 20.
    Turnbull, D., Liu, R., Barrington, L., Lanckriet, G.: A game-based approach for collecting semantic annotations of music. In: Proceedings International Conference on Music Information Retrieval (2007)Google Scholar
  21. 21.
    Law, E.L.M., von Ahn, L., Dannenberg, R.B., Crawford, M.: TagATune: A game for music and sound annotation. In: Proceedings International Conference on Music Information Retrieval (2007)Google Scholar
  22. 22.
    van Zwol, R., Garcia, L., Ramirez, G., Sigurbjörnsson, B., Labad, M.: Video tag game. In: Proceedings International World Wide Web Conference (2008)Google Scholar
  23. 23.
    Gligorov, R., Baltussen, L.B., van Ossenbruggen, J., Aroyo, L., Brinkerink, M., Oomen, J., van Ees, A.: Towards integration of end-user tags with professional annotations. In: Proceedings International Web Science Conference (2010)Google Scholar
  24. 24.
    Gonçalves, D., Jesus, R., Grangeiro, F., Romao, T., Correia, N.: Tag around: a 3D gesture game for image annotation. In: Proceedings International Conference on Advances in Computer Entertainment Technology, pp. 259–262 (2008)Google Scholar
  25. 25.
    Seneviratne, L., Izquierdo, E.: An interactive framework for image annotation through gaming. In: Proceedings ACM International Conference on Multimedia Information Retrieval, pp. 517–526 (2010)Google Scholar
  26. 26.
    Yao, B., Yang, X., Zhu, S.-C.: Introduction to a large-scale general purpose ground truth database: Methodology, annotation tool and benchmarks. In: Energy Minimization Methods in Computer Vision and Pattern Recognition, vol. 4679, LNCS, pp. 169–183, Springer (2007)Google Scholar
  27. 27.
    Petridis, K., Anastasopoulos, D., Saathoff, C., Timmermann, N., Kompatsiaris, Y., Staab, S.: M-ontomat-annotizer: Image annotation linking ontologies and multimedia low-level features. In: B. Gabrys, R.J. Howlett, and L.C. Jain, editors, KES (3), vol. 4253 of LNCS, pp. 633–640, Springer (2006)Google Scholar
  28. 28.
    Halaschek-Wiener, C., Golbeck, J., Schain, A., Grove, M., Parsia, B., Hendler, J.: Photostuff—an image annotation tool for the semantic web. In: Proceedings International Semantic Web Conference (2005)Google Scholar
  29. 29.
    Hollink, L., Nguyen, G., Schreiber, G., Wielemaker, J., Wielinga, B., Worring, M.: Adding spatial semantics to image annotations. In: Proceedings International Workshop on Knowledge Markup and Semantic Annotation at ISWC (2004)Google Scholar
  30. 30.
  31. 31.
    Volkmer, T., Smith, J.R., Natsev, A.(P.): A web-based system for collaborative annotation of large image and video collections: an evaluation and user study. In: Proceedings ACM international conference on Multimedia, pp. 892–901 (2005)Google Scholar
  32. 32.
    Marlow, C., Naaman, M., Boyd, D., Davis, M.: Ht06, tagging paper, taxonomy, flickr, academic article, to read. In: Proceedings International Conference on Hypertext and Hypermedia, pp. 31–40 (2006)Google Scholar
  33. 33.
    Ames, M., Naaman, M.: Why we tag: Motivations for annotation in mobile and online media. In: Proceedings SIGCHI Conference on Human Factors in Computing Systems, pp. 971–980 (2007)Google Scholar
  34. 34.
    Sorokin, A., Forsyth, D.: Utility data annotation with amazon mechanical turk. In: Proceedings IEEE Computer Vision and Pattern Recognition Workshops (2008)Google Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  1. 1.University of AmsterdamAmsterdamThe Netherlands

Personalised recommendations