Abstract
Canale introduces learning from a multimodal socio-semiotic perspective, arguing this has profound theoretical and ethical implications. In light of this, Canale explores some of the many connections between technology, meaning-making and multimodality in the understanding that new technology plays a fundamental role in current formal education and that the interaction, communication and learning that take place with such technology in schools needs to be approached socio-semiotically and multimodally. Drawing on classroom examples, he discusses the benefits of approaching learning from this multimodal socio-semiotic perspective as an alternative to long-ingrained ideologies of learning which permeate many school and classroom practices and which obscure much of the meaning-making and learning that takes place inside (and outside) the school walls.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
- 2.
- 3.
- 4.
Often times in additional language learning theories, non-verbal resources have been considered to be used strategically to compensate for the lack of verbal resources in communication, as a sort of compensatory strategy. For instance, in the traditional definition of communicative competence (Canale and Swain 1980; Canale 1983), strategic competence is a sub-component which accounts for verbal and non-verbal resources used to compensate for communication problems in the target language, pointing to what the individual hasn’t still acquired or learned.
References
Adami, E., & Kress, G. (2014). Introduction: Multimodality, Meaning Making and the Issue of Text. Text & Talk, 34(3), 231–237. https://doi.org/10.1515/text-2014-0007
Ahearn, L. (2001). Language and Agency. Annual Review of Anthropology, 30, 109–137.
Alexander, B. (2006). A Threat to Professional Identity? The Resistance to Computer Mediated Teaching. In M. Hanrahan & D. L. Madsen (Eds.), Teaching, Technology, Textuality. Approaches to New Media (pp. 27–35). New York: Palgrave Macmillan.
Angenot, M. (1985). Critique de la raison sémiotique. Montréal: Presses de L´Université de Montréal.
Apple, M. W. (2003). The State and the Politics of Knowledge. In M. W. Apple (Ed.), The State and the Politics of Knowledge (pp. 1–23). New York: Routledge Falmer.
Apple, M. W., & Jungck, S. (1998). “You Don’t Have to be a Teacher to Teach this Unit”: Teaching, Technology and Control in the Classroom. In H. Bromley & M. W. Apple (Eds.), Education/Technology/Power. Educational Computing as a Social Practice (pp. 133–155). New York: State University of New York Press.
Barthes, R. (1977). Image, Music, Text. London: Fontana.
Bezemer, J., & Kress, G. (2008). Writing in Multimodal Texts: A Social Semiotic Account of Design for Learning. Written Communication, 25(2), 165–195. https://doi.org/10.1177/0741088307313177
Bezemer, J., & Kress, G. (2015). The Textbook in a Changing Multimodal Landscape. In N. M. Klug & H. Stöckl (Eds.), Language in Multimodal Contexts (pp. 21–28). New York: De Gruyter.
Bezemer, J., & Kress, G. (2016). Multimodality, Learning and Communication. A Social Semiotic Framework. London and New York: Routledge.
Block, D. (2013). Moving Beyond “Lingualism”: Multilingual Embodiment and Multimodality in SLA. In S. May (Ed.), The Multilingual Turn. Implications for SLA, TESOL and Bilingual Education (pp. 54–77). New York and London: Routledge.
Blyth, C. (2009). From Textbook to Online Materials: The Changing Ecology of Foreign Language Publishing in an Era of ICT. In M. Evans (Ed.), Foreign Language Learning with Digital Technologies (pp. 174–202). London and New York: Continuum.
Boody, R. M. (2001). On the Relationship of Education and Technology. In R. Muffoletto (Ed.), Education and Technology: Critical and Reflective Practices (pp. 5–22). Cresskill, NJ: Hampton Press.
Bowen, T., & Whithaus, C. E. (2013). Multimodal Literacies and Emerging Genres. Pittsburgh: University of Pittsburgh Press.
Bromley, H., & Apple, M. W. (Eds.). (1998). Education/Technology/Power. Educational Computing as a Social Practice. New York: State University of New York Press.
Canale, M. (1983). From Communicative Competence to Communicative Language Pedagogy. In J. C. Richards & R. W. Schmidt (Eds.), Language and Communication (pp. 2–27). London: Longman.
Canale, M., & Swain, M. (1980). Theoretical Basis of Communicative Approaches to Second Language Teaching and Testing. Applied Linguistics, 1(1), 1–47. https://doi.org/10.1093/applin/I.1.1
Christie, F. (2002). Classroom Discourse Analysis. A Functional Perspective. London and New York: Continuum.
Dowding, K. (Ed.). (2011). Encyclopedia of Power. London and Washington, DC: Sage.
Dreyfus, S. J., Humphrey, S., Mahbob, A., & Martin, J. R. (2016). Genre Pedagogy in Higher Education. The SLATE Project. London: Palgrave Macmillan.
Engeström, Y. (1987). Learning by Expanding. An Activity-Theoretical Approach to Developmental Research. Helsinki: Orienta-Konsuiltit.
Engeström, Y. (2001). Expansive Learning at Work: Toward an Activity Theoretical Reconceptualization. Journal of Education and Work, 14(1), 133–156. https://doi.org/10.1080/13639080020028747
Fairclough, N. (2003). Analysing Discourse. London and New York: Routledge.
Gee, J. P. (2000). New People in New Worlds: Networks, the New Capitalism and Schools. In B. Cope & M. Kalantzis (Eds.), Multiliteracies. Literacy Learning and the Design of Social Futures (pp. 41–66). London and New York: Routledge.
Gee, J. P. (2007). What Video Games have to Teach Us about Learning and Literacy. New York: Palgrave Macmillan.
Giddens, A. (1984). The Constitution of Society: Outline of the Theory of Structuration. Berkeley: University of California Press.
Guerrettatz, A. M., & Johnston, B. (2013). Materials in the Classroom Ecology. The Modern Language Journal, 97(3), 779–796. https://doi.org/10.1111/j.1540-4781.2013.12027.x
Hakkarainen, K., Paavola, S., Kangas, K., & Seitamma-Hakkarainen, P. (2013). Sociocultural Perspectives on Collaborative Learning: Toward Collaborative Knowledge Creation. In C. E. Hmelo-Silver, C. A. Chinn, C. K. K. Chan, & A. O’Donnell (Eds.), The International Handbook of Collaborative Learning (pp. 57–73). New York: Routledge.
Halliday, M. A. K. (1974/2004). A Sociosemiotic Perspective on Language Development. In J. Webster (Ed.), The Language of Early Childhood (pp. 90–111). London and New York: Continuum.
Halliday, M. A. K. (1993). Towards a Language-based Theory of Learning. Linguistics and Education, 5, 93–116. https://doi.org/10.1016/0898-5898(93)90026-7
Halliday, M. A. K. (1998/2004). Representing the Child as a Semiotic Being. In J. Webster (Ed.), The Language of Early Childhood (pp. 6–26). London and New York: Continuum.
Halliday, M. A. K. (2001). El lenguaje como semiótica social. Buenos Aires: Fondo de Cultura Económica.
Halliday, M. A. K. (2014). Introduction to Functional Grammar. London and New York: Routledge.
Hasan, R. (1984). What Kind of Resource is Language? Australian Review of Applied Linguistics, 7, 57–86.
Hodge, B. (2017). Social Semiotics for a Complex World. Cambridge: Polity Press.
Hodge, R., & Kress, G. (1988). Social Semiotics. Cambridge: Polity Press.
Jewitt, C. (2006). Technology, Literacy and Learning. A Multimodal Approach. London and New York: Routledge.
Jewitt, C. (2008). Multimodality and Literacy in School Classrooms. Review of Research in Education, 32(1), 241–267.
Jewitt, C. (2011). An Introduction to Multimodality. In C. Jewitt (Ed.), The Routledge Handbook of Multimodal Analysis (pp. 14–27). London and New York: Routledge.
Jewitt, C. (2013). Multimodality and Digital Technologies in the Classroom. In I. de Saint-Georges & J. J. Weber (Eds.), Multilingualism and Multimodality. Current Challenges for Educational Studies (pp. 141–152). Rotterdam: Sense.
Jewitt, C. (2014). Multimodality, “Reading” and “Writing” for the 21st Century. In C. Lutkewitte (Ed.), Multimodal Composition. A Critical Sourcebook (pp. 309–324). Boston and New York: Bedford / St. Martin’s.
Kalantzis, M., & Cope, B. (2000). Changing the Role of Schools. In B. Cope & M. Kalantzis (Eds.), Multiliteracies. Literacy Learning and the Design of Social Futures (pp. 117–144). London and New York: Routledge.
Kalantzis, M., Cope, B., & Cloonan, A. (2010). A Multiliteracies Perspective on the New Literacies. In E. A. Baker (Ed.), The New Literacies. Multiple Perspectives on Research and Practice (pp. 61–87). New York and London: Guilford.
Kaptelinin, V., & Nardi, B. A. (2006). Acting with Technology. Activity Theory and Interaction Design. Cambridge: MIT Press.
Kress, G. (1993). Against Arbitrariness: The Social Production of the Sign as a Foundational Issue in Critical Discourse Analysis. Discourse & Society, 4(2), 169–191. https://doi.org/10.1177/0957926593004002003
Kress, G. (2000a). Multimodality: Challenges to Thinking about Language. TESOL Quarterly, 34(2), 337–340. https://doi.org/10.2307/3587959
Kress, G. (2000b). Design as Transformation: New Theories of Meaning. In B. Cope & M. Kalantzis (Eds.), Multiliteracies: Learning Literacy and the Design of Social Futures (pp. 153–161). Melbourne: Macmillan.
Kress, G. (2003). Literacy in the New Media Age. London: Routledge.
Kress, G. (2004). Learning, a Semiotic View in the Context of Digital Technologies. In A. Brown & N. Davis (Eds.), Digital Technology, Communities & Education (pp. 15–39). New York: Routledge Falmer.
Kress, G. (2005). Gains and Losses: New Forms of Texts, Knowledge, and Learning. Computers and Composition, 22(1), 5–22. https://doi.org/10.1016/j.compcom.2004.12.004
Kress, G. (2010). Multimodality. A Sociosemiotic Approach to Contemporary Communication. London: Routledge.
Kress, G. (2011). What is Mode? In C. Jewitt (Ed.), The Routledge Handbook of Multimodal Analysis (pp. 54–67). London and New York: Routledge.
Kress, G. (2013). Recognizing Learning. A Perspective from a Social Semiotic Theory of Multimodality. In I. de Saint-Georges & J. J. Weber (Eds.), Multilingualism and Multimodality. Current Challenges for Educational Studies (pp. 120–140). Rotterdam: Sense.
Kress, G. (2015a). Designing Meaning. Social Semiotic Multimodality Seen in Relation to Ethnographic Research. In S. Bollig, M. S. Honig, S. Neumann, & C. Seele (Eds.), MultiPluriTrans in Educational Ethnography. Approaching the Multimodality, Plurality and Translocality of Educational Realities (pp. 213–233). Verlag: Transcript.
Kress, G. (2015b). Semiotic Work. Applied Linguistics and a Social Semiotic Account of Multimodality. AILA Review, 28, 49–71. https://doi.org/10.1075/aila.28.03kre
Kress, G., Jewitt, C., Bourne, J., Franks, A., Hardcastle, J., Jones, K., & Reid, E. (2005). English in Urban Classrooms. A Multimodal Perspective on Teaching and Learning. London: Routledge Farmar.
Kress, G., Jewitt, C., Ogborn, J., & Tsatsarelis, C. (2001). Multimodal Teaching and Learning: The Rhetorics of the Science Classroom. London: Continuum.
Kress, G., & Selander, S. (2012). Multimodal Design, Learning and Cultures of Recognition. Internet and Higher Education, 15(4), 265–268. https://doi.org/10.1016/j.iheduc.2011.12.003
Kress, G., & van Leeuwen, T. (2001). Multimodal Discourse. The Modes and Media of Contemporary Communication. London: Arnold.
Kress, G., & van Leeuwen, T. (2006). Reading Images. The Grammar of Visual Design. London and New York: Routledge.
Latour, B. (1987). Science in Action. How to Follow Scientists and Engineers through Society. Cambridge, MA: Harvard University Press.
Latour, B. (2006). Reassembling the Social. An Introduction to Actor-Network Theory. Oxford: Oxford University Press.
Lawn, M. (1999). Designing Teaching. The Classroom as a Technology. In I. Grosvenor, M. Lawn, & K. Rousmaniere (Eds.), Silences & Images. The Social History of the Classroom (pp. 65–82). New York: Peter Lang.
Leander, K., & Vasudevan, L. (2009). Multimodality and Mobile Culture. In C. Jewitt (Ed.), Handbook of Multimodal Analysis (pp. 127–139). London: Routledge.
LeBaron, C. (2002). Technology Does not Exist Independent of Its Use. In T. Koschmann, R. Hall, & N. Miyake (Eds.), CSCL 2: Carrying Forward the Conversation (pp. 433–439). Mahwah: Lawrence Erlbaum.
Lemke, J. (1990). Talking Science: Language, Learning, and Values. Norwood: Ablex Publishing.
Lemke, J. (1995a). Textual Politics. Discourse and Social Dynamics. London and Bristol: Taylor & Francis.
Lemke, J. L. (1995b). Intertextuality and Text Semantics. In M. Gregory & P. Fries (Eds.), Discourse in Society: Functional Perspectives (pp. 85–114). Norwood: Ablex Publishing.
Lemke, J. L. (1998). Multiplying Meaning: Visual and Verbal Semiotics in Scientific Text. In J. R. Martin & R. Veel (Eds.), Reading Science (pp. 87–113). London: Routledge.
Lemke, J. (2000). Across the Scales of Time: Artifacts, Activities, and Meanings in Ecosocial Systems. Mind, Culture, and Activity, 7(4), 273–290. https://doi.org/10.1207/S15327884MCA0704_03
Lemke, J. (2011). Multimodality, Identity, and Time. In C. Jewitt (Ed.), The Routledge Handbook of Multimodal Analysis (pp. 140–150). London: Routledge.
Lin, A. (2012). Multilingual and Multimodal Resources in Genre-Based Pedagogical Approaches to L2 English Content Classrooms. In C. Leung & B. V. Street (Eds.), English. A Changing Medium for Education (pp. 79–103). Bristol, Buffalo and Toronto: Multilingual Matters.
Lotherington, H., & Ronda, N. (2012). Multimodal Literacies and Assessment: Uncharted Challenges in the English Classroom. In C. Leung & B. V. Street (Eds.), English—A Changing Medium for Education (pp. 104–128). Clevedon: Multilingual Matters.
Luke, C. (2000). Cyber-schooling and Technological Change: Multiliteracies for New Times. In B. Cope & M. Kalantzis (Eds.), Multiliteracies. Literacy Learning and the Design of Social Futures (pp. 67–88). London and New York: Routledge.
Luke, C., de Castell, S., & Luke, A. (1989). Beyond Criticism: The Authority of the School Textbook. In S. de Castell, A. Luke, & C. Luke (Eds.), Language Authority and Criticism. Readings on the School Textbook (pp. 245–260). London and New York: Falmer.
Machin, D., & Mayr, A. (2012). How to do Critical Discourse Analysis. London: Sage.
Machin, D., & van Leeuwen, T. (2007). Global Media Discourse. A Critical Introduction. London and New York: Routledge.
Manghi, D. (Ed.). (2017). La complejidad de la interacción en el aula, reconociendo significados que transforman. Valparaíso: Ediciones Universitarias de Valparaíso.
Manghi Haquin, D. (2013). Representación y comunicación del conocimiento en Educación Media: análisis multimodal del discurso de materiales utilizados para la enseñanza escolar de la historia y de la biología. Onomázwin, 27, 35–52.
Martin, J. R. (2000). Grammar Meets Genre—Reflections on the ‘Sydney School’. Arts: The Journal of the Sydney University Arts Association, 22, 47–95.
Martin, J. R., & Rose, D. (2007). Working with Discourse. London and New York: Continuum.
Martin, J. R., & Rose, D. (2008). Genre Relations: Mapping Culture. London: Equinox.
Matthiessen, C. I. M. M. (2014). The Multimodal Page: A Systematic Functional Exploration. In T. D. Royce & W. L. Bowcher (Eds.), New Directions in the Analysis of Multimodal Discourse (pp. 1–62). London and New York: Routledge.
Matthiessen, C. I. M. M., Teruya, K., & Lam, M. (2010). Keyterms in Systemic Functional Linguistics. London and New York: Continuum.
Miettinen, R. (2009). The Riddle of Things. Activity Theory and Actor Network Theory as Approaches to Studying Innovations. Mind, Culture and Activity, 6(3), 170–195. https://doi.org/10.1080/10749039909524725
Miller, M. S., & McVee, M. (Eds.). (2012). Multimodal Composing in Classrooms. Learning and Teaching for the Digital World. New York and London: Routledge.
Muffoletto, R. (2001). The Need for Critical Theory and Reflective Practices in Educational Technology. In R. Muffoletto (Ed.), Education and Technology. Critical and Reflective Practices (pp. 285–299). Cresskill: Hampton Press.
Norris, S. (2011). Modal density and Modal Configurations: Multimodal Actions. In C. Jewitt (Ed.), The Routledge Handbook of Multimodal Analysis (pp. 78–90). London and New York: Routledge.
O’Halloran, K. (2012). Análisis del discurso multimodal. Revista de la Asociación Latinoamericana de Estudios del Discurso, 12(1), 75–97.
Painter, C., Martin, J. R., & Unsworth, L. (2012). Reading Visual Narratives. Image Analysis in Children’s Picture Books. Sheffield: Equinox.
Resnik, B. L., Pontecorvo, C., & Säljö, R. (1997). Discourse, Tools and Reasoning. In B. L. Resnik, R. Säljö, & C. Pontecorvo (Eds.), Discourse, Tools and Reasoning. Essays on Situated Cognition (pp. 1–21). Berlin: Springer.
Röhl, T. (2015). Transsituating Education. Educational Artifacts in the Classroom and Beyond. In S. Bollig, M. S. Honig, S. Neumann, & C. Seele (Eds.), MultiPluriTrans in Educational Ethnography. Approaching the Multimodality, Plurality and Translocality of Educational Realities (pp. 122–139). Verlag: Transcript.
Rose, D. (2010). Genre in the Sydney School. In J. P. Gee & M. Handford (Eds.), The Routledge Handbook of Discourse Analysis. London: Routledge.
Rothery, J., & Stenglin, M. (1995). Exploring Literacy in School English. Sydney: Metropolitan East Disadvantaged Schools Program.
Royce, T. (2002). Multimodality in the TESOL Classroom: Exploring Visual-Verbal Synergy. TESOL Quarterly, 36(2), 191–205. https://doi.org/10.2307/3588330
Royce, T. (2007). Multimodal Communicative Competence. In T. Royce & W. Bowcher (Eds.), New Directions in the Analysis of Multimodal Discourse (pp. 361–390). Mahwah: Lawrence Erlbaum Associates.
Sannino, A., Daniels, H., & Gutiérrez, K. D. (Eds.). (2009). Learning and Expanding with Activity Theory. Cambridge: Cambridge University Press.
Saussure, F. (1967). Cours de Linguistique Générale. Paris: Grande Bibliotèque Payot.
Schatzki, T. R. (2001). Introduction: Practice Theory. In T. R. Schatzki, K. K. Cetina, & E. von Savigny (Eds.), The Practice Turn in Contemporary Theory (pp. 1–14). London: Routledge.
Schnotz, W. (1997). Strategy-Specific Information Access in Knowledge Acquisition from Hypertext. In B. L. Resnik, R. Säljö, & C. Pontecorvo (Eds.), Discourse, Tools and Reasoning. Essays on Situated Cognition (pp. 336–359). Berlin: Springer.
Schofield, W. J. (1995). Computers and Classroom Culture. New York: Cambridge University Press.
Scollon, R., & Levine, P. (2004). Multimodal Discourse Analysis as the Confluence of Discourse and Technology. In P. Levine & R. Scollon (Eds.), Discourse & Technology. Multimodal Discourse Analysis (pp. 1–6). Washington, DC: Georgetown University Press.
Serafini, F. (2014). Reading the Visual. An Introduction to Teaching Multimodal Literacy. New York: Teachers College Press.
Setzer, V. W., & Monke, L. (2001). Challenging the Application: An Alternative View on Why, When and How Computers Should Be Used in Education. In R. Muffoletto (Ed.), Education and Technology. Critical and Reflective Practices (pp. 141–173). Cresskill: Hampton Press.
Stahl, G., Koschmann, T., & Suthers, D. (2006). Computer-Supported Collaborative Learning: An Historical Perspective. In R. K. Sawyer (Ed.), Cambridge Handbook of the Learning Sciences (pp. 409–426). Cambridge: Cambridge University Press.
Stein, P. (2008). Multimodal Pedagogies in Diverse Classrooms. Representations, Rights and Resources. London and New York: Routledge.
Stockl, H. (2007). In Between Modes: Language and Image in Printed Media. In E. Ventola, C. Charles, & M. Kaltenbacher (Eds.), Perspectives on Multimodality (pp. 9–30). Amsterdam: John Benjamins.
Sung, Y. K., & Apple, M. (2003). Democracy, Technology, and Curriculum: Lessons from the Critical Practices of Korean Teachers. In M. W. Apple (Ed.), The State and the Politics of Knowledge (pp. 177–191). New York: Routledge Falmer.
The New London Group. (1996). A Pedagogy of Multiliteracies: Designing Social Futures. Harvard Educational Review, 66(1), 60–92.
Tudor, I. (2001). The Dynamics of the Language Classroom. Cambridge: Cambridge University Press.
Unsworth, L. (2008). Multimodal Semiotic Analyses and Education. In L. Unsworth (Ed.), Multimodal Semiotics. Functional Analysis in Contexts of Education (pp. 1–13). London and New York: Continuum.
van Leeuwen, T. (2004). Ten Reasons Why Linguists Should Pay Attention to Visual Communication. In P. Levine & R. Scollon (Eds.), Discourse & Technology. Multimodal Discourse Analysis (pp. 7–19). Washington, DC: Georgetown University Press.
van Leeuwen, T. (2005). Introducing Social Semiotics. London and New York: Routledge.
van Lier, L. (2004a). The Semiotics and Ecology of Language Learning. Perception, Voice, Identity and Democracy. Utbildning & Demokrati, 13(3), 79–103.
van Lier, L. (2004b). The Ecology and Semiotics of Language Learning. A Sociocultural Perspective. New York: Kluwer Academic Publishers.
van Oers, B., Wardekker, W., Elbers, E., & van der Veer, R. (Eds.). (2008). The Transformation of Learning. Advances in Cultural-Historical Activity Theory. Cambridge: Cambridge University Press.
Vološinov, V. N. (1973). Marxism and the Philosophy of Language. New York and London: Seminar Press.
Vygotsky, L. S. (1978). Mind in Society: The Development of Higher Psychological Processes. Cambridge: Harvard University Press.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2019 The Author(s)
About this chapter
Cite this chapter
Canale, G. (2019). Toward a Multimodal Socio-Semiotic Account of Learning. In: Technology, Multimodality and Learning. Palgrave Studies in Educational Media. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-030-21795-2_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-21795-2_3
Published:
Publisher Name: Palgrave Macmillan, Cham
Print ISBN: 978-3-030-21794-5
Online ISBN: 978-3-030-21795-2
eBook Packages: EducationEducation (R0)