Language Resources and Evaluation

, Volume 50, Issue 3, pp 443–474 | Cite as

Image selection and annotation for an environmental knowledge base

  • Arianne Reimerink
  • Pilar León-Araúz
  • Pamela Faber
Original Paper


Images play an important role in the representation and acquisition of specialized knowledge. Not surprisingly, terminological knowledge bases (TKBs) often include images as a way to enhance the information in concept entries. However, the selection of these images should not be random, but rather based on specific guidelines that take into account the type and nature of the concept being described. This paper presents a proposal on how to combine the features of images with the conceptual propositions in EcoLexicon, a multilingual TKB on the environment. This proposal is based on the following: (1) the combinatory possibilities of concept types; (2) image types, such as photographs, drawings and flow charts; (3) morphological features or visual knowledge patterns (VKPs), such as labels, colours, arrows, and their effect on the functional nature of each image type. Currently, images are stored in association with concept entries according to the semantic content of their definitions, but they are not described or annotated according to the parameters that guided their selection, which would undoubtedly contribute to the systematization and automatization of the process. First, the images included in EcoLexicon were analyzed in terms of their adequateness, the semantic relations expressed, the concept types and their VKPs. Then, with these data, guidelines for image selection and annotation were created. The final aim is twofold: (1) to systematize the selection of images and (2) to start annotating old and new images so that the system can automatically allocate them in different concept entries based on shared conceptual propositions.


Knowledge representation Image selection Image annotation EcoLexicon 



This research was carried out within the framework of the project RECORD [Knowledge Representation in Dynamic Networks, FFI2011-22397] and the project CONTENT [Cognitive and Neurological Bases for Terminology-enhanced Translation], both funded by the Spanish Ministry of Economy and Competitiveness.


  1. Anglin, G., Vaez, H., & Cunningham, K. (2004). Visual representations and learning: The role of static and animated graphics. In D. Jonassen (Ed.), Handbook of research on educational communications and technology (pp. 755–794). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  2. Barriuso, A., & Torralba, A. (2012) Notes on image annotation. arXiv:1210.3448 [cs.CV].
  3. Barsalou, L. W. (2003). Situated simulation in the human conceptual system. Language and Cognitive Processes, 18, 513–562.CrossRefGoogle Scholar
  4. Darian, S. (2001). More than meets the eye: The role of visuals in science textbooks. LSP & Professional Communication, 1(1), 10–36.Google Scholar
  5. Di, W., Wah, C., Bhardwaj, A., Piramuthu, R., & Sudaresan, N. (2013). Style finder: Fine-grained clothing style recognition and retrieval. In Third IEEE international workshop on mobile vision, Portland, OR, USA.Google Scholar
  6. Faber, P. (Ed.). (2012). A cognitive linguistics view of terminology and specialized language. Berlin: De Gruyter Mouton.Google Scholar
  7. Faber, P., León Araúz, P., Prieto Velasco, J. A., & Reimerink, A. (2007). Linking images and words: The description of specialized concepts. International Journal of Lexicography, 20(1), 39–65. doi: 10.1093/ijl/ecl038.CrossRefGoogle Scholar
  8. Faber, P., Montero Martínez, S., Castro Prieto, M. R., Senso Ruiz, J., Prieto Velasco, J. A., León Araúz, P., et al. (2006). Process-oriented terminology management in the domain of coastal engineering. Terminology, 12(2), 189–213.CrossRefGoogle Scholar
  9. Jeon, J., Lavrenko, V., & Manmatha, R. (2003). Automatic image annotation and retrieval using cross-media relevance models. In Proceedings of the 26th annual international ACM SIGIR conference on research and development in information retrieval, Toronto, Canada, pp. 119–126.Google Scholar
  10. Kageura, K. (1997). Multifaceted/multidimensional concept systems. In S. E. Wright & G. Budin (Eds.), Handbook of terminology management: Basic aspects of terminology management (pp. 119–132). Amsterdam: John Benjamins.CrossRefGoogle Scholar
  11. León Araúz, P. (2009). Representación multidimensional del conocimiento especializado: el uso de marcos desde la macroestructura hasta la microestructura. Ph.D. thesis, University of Granada.Google Scholar
  12. León Araúz, P., & Faber, P. (2010). Natural and contextual constraints for domain-specific relations. In V. Barbu Mititelu, V. Pekar, E. Barbu (Eds.), Proceedings of the workshop semantic relations, theory and applications, Valletta, pp. 12–17.Google Scholar
  13. León Araúz, P., Reimerink, A., & Faber, P. (2009). Knowledge extraction on multidimensional concepts: Corpus pattern analysis (CPA) and concordances. In 8ème Conférence Internationale Terminologie et Intelligence Artificielle, Toulouse.Google Scholar
  14. León-Araúz, P., Reimerink, A., & Faber, P. (2013). Multidimensional and multimodal information in EcoLexicon. In A. Przepiórkowski, M. Piasecki, K. Jassem, & P. Fluglewicz (Eds.), Computational linguistics (pp. 143–161). Berlin: Springer.CrossRefGoogle Scholar
  15. Levie, W. H., & Lentz, R. (1982). Effects of text illustrations: A review of research. Educational Communication and Technology Journal, 30, 195–232.Google Scholar
  16. Li, J., & Wang, J. Z. (2008). Real-time computerized annotation of pictures. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(6), 985–1002.CrossRefGoogle Scholar
  17. Mayer, R. E., & Anderson, R. B. (1992). The instructive animation: Helping students build connections between words and pictures in multimedia learning. Journal of Educational Psychology, 84(4), 715–726.Google Scholar
  18. Mei, T., Wang, Y., Hua, X. S., Gong, S., & Li, S. (2008). Coherent image annotation by learning semantic distance. In IEEE conference on computer vision and pattern recognition, Piscataway, NJ, USA, pp. 1–8.Google Scholar
  19. Monterde Rey, A. M. (2002). Terminología: estudio de las distintas formas de representación conceptual en textos técnicos y su relación con la traducción. In Actas de las II Jornadas de Jóvenes Traductores (pp. 147–156). Las Palmas de G. C: Servicio de Publicaciones de la Universidad de Las Palmas de G. C.Google Scholar
  20. Montero-Martínez, S., & García de Quesada, M. (2004). Designing a corpus-based grammar for pragmatic terminographic definitions. Journal of Pragmatics, 36(2), 265–291.CrossRefGoogle Scholar
  21. Murphy, M. L. (2003). Semantic relations and the lexicon. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  22. Nöth, W. (2001). Word and image: Intermedial aspects. Medien Pädagogik, 6.
  23. Paivio, A. (1971). Imagery and verbal processes. New York: Holt, Rinehart & Winston.Google Scholar
  24. Paivio, A. (1986). Mental representations: A dual-coding approach. New York: Oxford University Press.Google Scholar
  25. Parikh, D. (2013). Visual attributes for enhanced human-machine communication. In Allerton, pp. 1126–1127.Google Scholar
  26. Park, O., & Hopkins, R. (1993). Instructional conditions for using animated visual displays: A review. Instructional Science, 22, 1–24.Google Scholar
  27. Prieto Velasco, J. A. (2008). Información gráfica y grados de especialidad en el discurso científico-técnico: un estudio de corpus. Ph.D. thesis, University of Granada.Google Scholar
  28. Prieto Velasco, J. A., & Faber, P. (2012). Graphical information. In P. Faber (Ed.), A cognitive linguistics view of terminology and specialized language (pp. 225–248). Berlin: De Gruyter Mouton.Google Scholar
  29. Reimerink, A., & Faber, P. (2009). A frame-based knowledge base for the environment. In Proceedings of towards e-environment, Prague, pp. 629–636.Google Scholar
  30. Reimerink, A., García de Quesada, M., & Montero-Martínez, S. (2010). Contextual information in terminological knowledge bases: A multimodal approach. Journal of Pragmatics, 42(7), 1928–1950. doi: 10.1016/j.pragma.2009.12.008.CrossRefGoogle Scholar
  31. Rieber, L. P. (1994). Computers, graphics, and learning. Madison, WI: Brown & Benchmark.Google Scholar
  32. Roger, M. (2004). Multidimensionality in concepts systems: A bilingual textual perspective. Terminology, 10(2), 215–240.CrossRefGoogle Scholar
  33. Scholz, D. (2000). A solid sense of syntax. Erkenntnis, 52, 199–212.CrossRefGoogle Scholar
  34. Smeulder, A., Worring, M., Santini, S., Gupta, A., & Jain, R. (2000). Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12), 1349–1380.CrossRefGoogle Scholar
  35. Wenyin, L., Dumais, S., Sun, Y., Zhang, H. J., Czerwinski, M., & Field, B. (2001). Semi-automatic image annotation. In Proceedings of interact 2001: Conference on humancomputer interaction, pp. 326–333.Google Scholar
  36. Winston, M. E., Chaffin, R., & Herrmann, D. (1987). A taxonomy of part-whole relations. Cognitive Science, 11, 417–444.CrossRefGoogle Scholar
  37. Zitnick, C. L., & Parikh, D. (2013). Bringing semantics into focus using visual abstraction. In IEEE conference on computer vision and pattern recognition (CVPR), pp. 3009–3016.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  • Arianne Reimerink
    • 1
  • Pilar León-Araúz
    • 1
  • Pamela Faber
    • 1
  1. 1.University of GranadaGranadaSpain

Personalised recommendations