Researching Multimodality in Language and Education

Abstract

Multimodality has emerged out of relative obscurity over the past two decades as a term that has purchase in language and literacy research, theory, and practice. This chapter is concerned with research methods and approaches to the study of multimodality in language and education. Origins, early developments, and current uses of multimodality are discussed. Research methods that examine multimodal meaning-making operate across varied contexts and use diverse heuristics and research tools.

Multimodality explains communication as a combination of modes of representation and expression within text designs (with the term text referring to communicative acts beyond but including print or writing). The various multiple modes of expression can be visual (e.g., drawing, painting, video), print (e.g., books, newspapers, environmental print), gestural (e.g., miming, pointing, acting out), dramatic (e.g., role-playing, improvisation, formal acting), and oral (e.g., informal talk, public speaking) modes (Kress Multimodality. In: Cope B, Kalantzis M (eds) Multiliteracies: literacy learning and the design of social futures. Routledge, London, pp 182–202, 2000; Stein TESOL Q 32(3): 517–528, 1998). Kress often stands as a harbinger of multimodality with his theory and writings over the years, but this field can be traced to earlier theorists such as Halliday (with whom Kress studied).

Major research trends in response to early developments have emphasized design, digital and visual literacies, juxtapositions of home and school literacies, and interests in material culture, to name a few. Current explorations of embodiment, Deleuzian approaches, material culture, cultural geography, and gamification have resulted in innovative research methods. Challenges of multimodal applications to educational contexts that move beyond representation and that address earlier claims and issues of equity and social justice are described.