TagBag: Annotating a Foreign Language Lexical Resource with Pictures

  • Dmitry UstalovEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 542)


Such forms of art as photography or drawing may serve as a uniform language, which represents things that we can either see or imagine. Hence, it is reasonable to use such pictures in order to connect nouns of the natural languages by their meanings. In this paper a study of mapping noun images from an annotated collection to the word senses of a foreign language lexical resource through the usage of a bilingual dictionary has been conducted. In this study, the English-Russian dictionary by V.K. Mueller has been used to enhance the Yet Another RussNet synsets with Flickr photos.


Multimedia search Bilingual dictionary Image database Lexical ontology Natural language processing 



This work is supported by the Russian Foundation for the Humanities, project no. 13-04-12020 “New Open Electronic Thesaurus for Russian”, and by the Program of Government of the Russian Federation 02.A03.21.0006 on 27.08.2013. The URAN supercomputer located at the N.N. Krasovskii Institute of Mathematics and Mechanics of the Ural Branch of the Russian Academy of Sciences has been used to obtain the image collection. The author is grateful to those annotators who participated in the evaluation. He is also grateful to the anonymous referees who offered very useful comments on the present paper.


  1. 1.
    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 248–255 (2009)Google Scholar
  2. 2.
    Gelfenbein, I., et al.: Avtomaticheskij perevod semanticheskoj seti WORDNET na russkij yazyk. In: Proceedings of Dialog 2003 (2003) (in Russian)Google Scholar
  3. 3.
    Müller, H., Clough, P., Deselaers, T., Caputo, B. (eds.): ImageCLEF. The Information Retrieval Series, vol. 32. Springer, Heidelberg (2010)zbMATHGoogle Scholar
  4. 4.
    Joshi, D., Wang, J.Z., Li, J.: The story picturing engine–a system for automatic text illustration. ACM Trans. Multimedia Comput. Commun. Appl. 2, 68–89 (2006)CrossRefGoogle Scholar
  5. 5.
    Mihalcea, R., Leong, C.W.: Toward communicating simple sentences using pictorial representations. Mach. Trans. 22, 153–173 (2008)CrossRefGoogle Scholar
  6. 6.
    Reiter, K., Soderland, S., Etzioni, O.: Cross-lingual image search on the web. In: Proceedings of the Workshop on Cross-Lingual Information Access (20th International Joint Conference on Artificial Intelligence) (2007)Google Scholar
  7. 7.
    Trojahn, C., Quaresma, P., Vieira, R.: A framework for multilingual ontology mapping. In: Proceedings of the Sixth International Conference on Language Resources and Evaluation, LREC 2008, Marrakech. European Language Resources Association (2008)Google Scholar
  8. 8.
    Stampouli, A., Giannakidou, E., Vakali, A.: Tag disambiguation through flickr and wikipedia. In: Yoshikawa, M., Meng, X., Yumoto, T., Ma, Q., Sun, L., Watanabe, C. (eds.) DASFAA 2010. LNCS, vol. 6193, pp. 252–263. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  9. 9.
    Jiang, Y., Liu, J., Lu, H.: Chat with illustration. Multimedia Syst. 1–12 (2014).
  10. 10.
    Li, W., Zhuge, H.: Summarising news with texts and pictures. In: 10th International Conference on Semantics, Knowledge and Grids (SKG), pp. 100–107 (2014)Google Scholar
  11. 11.
    Braslavski, P., Ustalov, D., Mukhin, M.: A spinning wheel for YARN: user interface for a crowdsourced thesaurus. In: Proceedings of the Demonstrations at the 14th Conference of the European Chapter of the Association for Computational Linguistics, Gothenburg, pp. 101–104. Association for Computational Linguistics (2014)Google Scholar
  12. 12.
    Karger, D.R., Oh, S., Shah, D.: Budget-optimal task allocation for reliable crowdsourcing systems. Oper. Res. 62, 1–24 (2014)CrossRefzbMATHGoogle Scholar
  13. 13.
    Fleiss, J.L., Levin, B., Paik, M.C.: Statistical Methods for Rates and Proportions, 3rd edn. Wiley, Hoboken (2003)CrossRefzbMATHGoogle Scholar
  14. 14.
    Cheng, M.M., Zhang, G.X., Mitra, N.J., Huang, X., Hu, S.M.: Global contrast based salient region detection. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 409–416 (2011)Google Scholar
  15. 15.
    von Ahn, L., Dabbish, L.: Labeling images with a computer game. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2004, pp. 319–326. ACM, New York (2004)Google Scholar
  16. 16.
    Loukachevitch, N.: Thesauri for Information Retrieval Tasks. MSU, Moscow (2011)Google Scholar
  17. 17.
    Ntoulas, A., Najork, M., Manasse, M., Fetterly, D.: Detecting spam web pages through content analysis. In: Proceedings of the 15th International Conference on World Wide Web, WWW 2006, pp. 83–92. ACM, New York (2006)Google Scholar
  18. 18.
    Flati, T., Navigli, R.: The CQC algorithm: cycling in graphs to semantically enrich and enhance a bilingual dictionary. J. Artif. Int. Res. 43, 135–171 (2012)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.N.N. Krasovskii Institute of Mathematics and MechanicsUral Branch of the Russian Academy of SciencesEkaterinburgRussia
  2. 2.Ural Federal UniversityEkaterinburgRussia
  3. 3.NLPubEkaterinburgRussia

Personalised recommendations