Abstract
In a seminal article, former linguist and semiotician Kress and his colleague van Leeuwen (Front pages: the (critical) analysis of newspaper layout. (In Bell A, Garrett, P (eds) Approaches to media discourse. Blackwell, London, pp 186–219, 1998) began to explain multimodal texts – texts which include a complex mixture of words and images. An interesting gap arising out of the work of Kress and his colleagues was their realization of multimodal texts and how meaning cannot be reduced to language only. In an attempt to address this gap, based on Michael Halliday’s (Language as social semiotic. Edward Arnold, London, 1978) functional model of language, a theory called multimodal social semiotic theory was formed by extending the theory of meaning beyond language (Kress G, van Leeuwen T, Reading images: the grammar of graphic design. Routledge, London, 1996). In other words, the meaning-making capacity of various modes (functions) such as music, sound, action, and visual communication is regarded as crucial as the language itself. In this paper, the foundations of the multimodal theory of communication and its implications for the field of learning design will be discussed.
Without integrative disciplines of understanding, communication, and action, there is little hope of sensibly extending knowledge beyond the library or laboratory in order to serve the purpose of enriching human life.
Buchanan (1992, p. 6)
In memory of Gunther R. Kress (1940–2019)
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Becoming Homeless: A Human Experience. (2017). Retrieved from Stanford University Virtual Human Interaction Lab, https://vhil.stanford.edu/becominghomeless/
Bezemer, J., & Jewitt, C. (2009). Multimodal analysis: Key issues. In L. Litosseliti (Ed.), Research methods in linguistics (pp. 180–197). London/New York: Continuum.
Bezemer, J., Jewitt, C., Diamantopoulou, S., Kress, G., & Mavers, D. (2012). Using a social semiotic approach to multimodality: Researching learning in Schools, Museums and Hospitals. NCRM Working Paper, NCRM, http://eprints.ncrm.ac.uk/2258/4/NCRM_working_paper_0112.pdf
Buchanan, R. (1992). Wicked problems in design thinking. Design Issues, 8(2), 5–21. https://doi.org/10.2307/1511637
Chomsky, N. (1965). Aspects of the theory of syntax. Cambridge, MA: MIT Press.
Halliday, M. A. K. (1978). Language as social semiotic. London: Edward Arnold.
Halliday, M. A. K., & Matthiessen, C. M. (2013). Halliday’s introduction to functional grammar. New York: Routledge.
Haratyan, F. (2011). Halliday’s SFL and social meaning. In 2nd international conference on Humanities, Historical and Social Sciences IPEDR, 17 (pp. 260–264).
Hodge, B. (2013). Kress on Kress: Using a method to read a life. In M. Böck & N. Pachler (Eds.), Multimodality and social semiotics: Communication, meaning making, and learning in the work of Gunther Kress (pp. 7–16). New York: Routledge.
Iedema, R. (2003). Multimodality, resemiotization: Extending the analysis of discourse as multi-semiotic practice. Visual Communication, 2(1), 29–57. https://doi.org/10.1177/1470357203002001751
Jewitt, C. (2006). Technology, literacy, learning: A multimodal approach. London: Routledge.
Kress, G. (1997). Before writing: Rethinking the paths to literacy. London: Routledge.
Kress, G. (1998). Visual and verbal modes of representation in electronically mediated communication: The potentials of new forms of text. In I. Snyder & M. Joyce (Eds.), Page to screen: Taking literacy into the electronic era (pp. 53–79). Sydney, Australia: Allen and Unwin.
Kress, G. (2003). Literacy in the new media age. London: Routledge.
Kress, G. (2010). Multimodality: A social semiotic approach to contemporary communication. New York: Routledge.
Kress, G. (2014a). The rhetorical work of shaping the semiotic world. In A. Archer & D. Newfield (Eds.), Multimodal approaches to research and pedagogy: Recognition, resources, and access (pp. 131–152). New York: Routledge/Taylor & Francis.
Kress, G. (2014b). Gains and losses: New forms of texts, knowledge, and learning. In C. Lutkewitte (Ed.), Multimodal composition: A critical sourcebook (pp. 283–301). Boston: Bedford/St. Martin’s.
Kress, G. (2019). Pedagogy as design: A social semiotic approach to learning as communication. Universitas Tarraconensis. Revista de Ciències de l’Educació, 1(2), 23–27. https://doi.org/10.17345/ute.2018.2.2488
Kress, G., & Bezemer, J. (2015). A social semiotic multimodal approach to learning. In E. Hargreaves & D. Scott (Eds.), The SAGE handbook of learning (pp. 155–168). London: Sage.
Kress, G., & van Leeuwen, T. (1996). Reading images: The grammar of graphic design. London: Routledge.
Kress, G., & van Leeuwen, T. (1998). Front pages: The (critical) analysis of newspaper layout. In A. Bell & P. Garrett (Eds.), Approaches to media discourse (pp. 186–219). London: Blackwell.
Kress, G., & van Leeuwen, T. (2006). Visual interaction. In A. Jaworski & N. Coupland (Eds.), The discourse reader (2nd ed., pp. 362–384). New York: Routledge.
Manghani, S., Piper, A., & Simons, J. (Eds.). (2006). Images: A reader. Thousand Oaks, CA: Sage.
Piattelli-Palmarini, M. (1980). Language and learning: The debate between Jean Piaget and Noam Chomsky. Cambridge, MA: Harvard University Press.
van Leeuwen, T. (2005). Introducing social semiotics. London/New York: Routledge.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Association for Educational Communications and Technology (AECT)
About this chapter
Cite this chapter
Saçak, B. (2021). Multimodal Social Semiotics and Learning Design: In Search of Interdisciplinarity. In: Hokanson, B., Exter, M., Grincewicz, A., Schmidt, M., Tawfik, A.A. (eds) Intersections Across Disciplines. Educational Communications and Technology: Issues and Innovations. Springer, Cham. https://doi.org/10.1007/978-3-030-53875-0_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-53875-0_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-53874-3
Online ISBN: 978-3-030-53875-0
eBook Packages: EducationEducation (R0)