Abstract
Modern communication environments have changed the cognitive patterns of individuals, who are now used to the interaction of information encoded in different semiotic modalities, especially visual and linguistic. Despite this, the main premise of Corpus Linguistics is still ruling: our perception of and experience with the world is conveyed in texts, which nowadays need to be studied from a multimodal perspective. Therefore, multimodal corpora are becoming extremely useful to extract specialized knowledge and explore the insights of specialized language and its relation to non-language-specific representations of knowledge. It is our assertion that the analysis of the image-text interface can help us understand the way visual and linguistic information converge in subject-field texts. In this article, we use Frame-based terminology to sketch a novel proposal to study images in a corpus rich in pictorial representations for their inclusion in a terminological resource on the environment. Our corpus-based approach provides the methodological underpinnings to create meaning within terminographic entries, thus facilitating specialized knowledge transfer and acquisition through images.
Similar content being viewed by others
Notes
The main objective of the MarcoCosta and EcoSistema research projects was to create a single information space for frame-based environmental data in the form of a visual thesaurus of specialized concepts in the domains of integrated coastal management and environmental science.
VariMed is a research project about the cognitive and cultural patterns underlying denominative variation in medical communication. It will result in a multimodal lexical resource in the field of health care with visual support for linguistic research, translation, and technical writing for knowledge dissemination.
References
Allwood, J. (2008). Multimodal corpora. In A. Lüdeling & M. Kytö (Eds.), Corpus linguistics: An international handbook (pp. 207–225). Berlin: Mouton de Gruyter.
Baddeley, A. D. (1986). Working memory. Oxford: Oxford University Press.
Baldry, A. P., & Thibault, P. J. (2006). Multimodal corpus linguistics. In G. Thompson & S. Hunston (Eds.), System and corpus exploring connections (pp. 164–183). London and New York: Equinox.
Baldry, A. P., & Thibault, P. J. (2008). Applications of multimodal concordances. Journal of Language and Communication Studies, 41, 11–41.
Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645.
Bowker, L. (2002). Computer-aided translation technology. A practical introduction. Ottawa: University of Ottawa Press.
Bowker, L., & Howkins, S. (2006). Variation in the organization of medical terms. Exploring some motivations for term choice. Terminology, 12(1), 79–110.
Carney, R. N., & Levin, J. R. (2002). Pictorial illustrations still improve students’ learning from text. Educational Psychology Review, 14(1), 5–26.
Faber, P. (2009). The cognitive shift in terminology and specialized translation. MonTI. Monografías de Traducción e Interpretación, no., 1, 107–134.
Faber, P. (2011). The dynamics of specialized knowledge representation: Simulational reconstruction or the perception–action interface. Terminology, 17(1), 9–29.
Faber, P., León Araúz, P., Prieto Velasco, J. A., & Reimerink, A. (2007). Linking images and words: The description of specialized concepts. International Journal of Lexicography, 20(1), 39–65.
Faber, P., Márquez Linares, C., & Vega, M. (2005). Framing terminology: A process-oriented approach. Meta: Journal des traducteurs/Meta: Translators’ Journal, 50(4).
Faber, P., Montero Martínez, S., Castro Prieto, M. R., Senso Ruiz, J., Prieto Velasco, J. A., León Araúz, P., et al. (2006). Process-oriented terminology management in the domain of coastal engineering. Terminology, 12(2), 189–213.
Fillmore, C. (1982). Frame semantics. In The Linguistic Society of Korea (Ed.), Linguistics in the morning calm (pp. 111–137). Seoul: Hanshin.
Fillmore, C. (1985). Frames and the semantics of understanding. Quaderni di Semantica, 6(2), 222–254.
Fillmore, C., & Atkins, B. (1992). Towards a frame-based organization of the lexicon: The semantics of RISK and its neighbors. In A. Lehrer & E. Kittay (Eds.), Frames, fields, and contrasts: New essays in semantics and lexical organization (pp. 75–102). Hillsdale: Lawrence Erlbaum.
Firth, J. R. (1957). A synopsis of linguistic theory 1930–1955. Studies in Linguistic Analysis (special volume of the Philological Society) (pp. 1–32).
Freixa, J. (2006). Causes of denominative variation in terminology: A typology proposal. Terminology, 12(1), 51–77.
Jewitt, C. (Ed.). (2009). Handbook of multimodal analysis. London: Routledge.
Kress, G., & Van Leeuwen, T. (2002a). Reading images: The grammar of visual design. London: Routledge.
Kress, G. R., & Van Leeuwen, T. (2002b). Multimodal discourse: The modes and media of contemporary communication. London: Edward Arnold.
León Araúz, P., & Faber, P. (2010). Natural and contextual constraints for domain-specific relations. In V. Barbu Mititelu, V. Pekar, & E. Barbu (Eds.), Proceedings of the Workshop Semantic Relations. Theory and Applications (pp. 12–17). Valletta.
Liu, Y., & O’Halloran, K. L. (2009). Intersemiotic texture: Analyzing cohesive devices between language and images. Social Semiotics, 19(4), 367–387.
López Rodríguez, C. I. (2001). Tipologías textuales y cohesión en la traducción biomédica inglés-español: un estudio de corpus. PhD Thesis. University of Granada. Available at http://www.ugr.es/~dpto_ti/profesores/cilr-tesis.html. Accessed Feb 2012.
López Rodríguez, C. I. (2009). Extracción y representación de conocimiento a partir de corpus. In E. Valero & E. Rambla Alcina (Eds.), Terminología y sociedad del conocimiento (pp. 341–379). Bern: Peter Lang.
López Rodríguez, C. I., Buendía Castro, M., & García Aragón, A. (2012). User needs to the test: Evaluating a terminological knowledge base on the environment by trainee translators. Jostrans. The Journal of Specialized Translation, 18, 57–76.
Louwerse, M. M., & Jeuniaux, P. (2010). The linguistic and embodied nature of conceptual processing. Cognition, 114, 96–104.
Marsh, E. E., & White, M. D. (2003). Taxonomy of relationships between images and text. Journal of Documentation, 59(6), 647–672.
Martinec, R., & Salway, A. (2005). A system for image-text relations in new (and old) media. Visual Communication, 4(3), 337–371.
McEnery, A., & Wilson, A. (2001). Corpus linguistics. Edinburgh: Edinburgh University Press.
Meyer, I., & Mackintosh, K. (1996). The corpus from a terminographer’s viewpoint. International Journal of Corpus Linguistics, 1(2), 257–285.
O’Halloran, K. L., & Smith B. A. (accepted for publication). Multimodal text analysis. In C. A. Chapelle (Ed.). Encyclopedia of applied linguistics. New Jersey: Wiley-Blackwell.
O’Halloran, K. L., Tan, S., Smith, B. A., & Podlasov, A. (2010). Challenges in designing digital interfaces for the study of multimodal phenomena. Information Design Journal, 18(1), 2–12.
Paivio, A. (1971). Imagery and verbal processes. New York: Holt, Rinehart, and Winston.
Prieto Velasco, J. A. (2008). Información gráfica y grados de especialidad en el discurso científico-técnico: un estudio de corpus. PhD Thesis, University of Granada. Available at: http://0-hera.ugr.es.adrastea.ugr.es/tesisugr/17491332.pdf. Accessed Feb 2012.
Prieto Velasco, J. A. (2009). Traducción e imagen: la información visual en textos especializados. Granada: Tragacanto.
Prieto Velasco, J. A., & López Rodríguez, C. I. (2009). Managing graphic information in terminological knowledge bases. Terminology, 15(2), 179–213.
Reimerink, A., García de Quesada, M., & Montero Martínez, S. (2010). Contextual information in terminological knowledge bases: A multimodal approach. Journal of Pragmatics, 42(7), 1928–1950.
Temmerman, R. (2000). Towards new ways of terminology description: The sociocognitive-approach. Amsterdam/Philadelphia: John Benjamins.
Temmerman, R., & Kerremans, K. (2003). Termontography: Ontology building and the sociocognitive approach to terminology description. Prague CIL17-conference.
Tercedor, M. I. (2011). The cognitive dynamics of terminological variation. Terminology, 17(2), 181–197.
Unsworth, L. (2008). Multimodal semiotics: Functional analysis in contexts of education. London and New York: Continuum.
Acknowledgments
This research is funded by the Spanish Ministry of Economy and Competitiveness under the project FFI2011-23120.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Prieto Velasco, J.A. A corpus-based approach to the multimodal analysis of specialized knowledge. Lang Resources & Evaluation 47, 399–423 (2013). https://doi.org/10.1007/s10579-012-9204-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10579-012-9204-1