This Special Issue of RISE is devoted to how theoretical perspectives in social semiotics help us understand science meaning-making when learners work with multiple representations. Articles here illustrate a range of research projects and theoretical aims in this contemporary and wide-ranging field. Our purpose here is to advance the field of meaning-making in science learning beyond cognitivist perspectives that, notably, have been incredibly influential. Social semiotics offers new opportunity to better understand meaning-making in the dynamic, multimodal environment that dominates contemporary contexts for science learning. The offerings here provide new insights for these different perspectives on theorizing multimodalities and multiple representations and we hope that these papers inspire new research across the many contexts for science learning.

The “multi” in multimodal and multiple refers to, possibly, several different forms of representation with which a reader or viewer engages. Learners may also construct their own representations to connect ideas in interesting ways using different modalities. With Kress and van Leeuwen (2001), we consider modalities to include visual, auditory, textual, gestural, and graphical, among others and typically, more than one of these is used in any situation where knowledge is represented. Different media, including digital media, offer affordances for meaning-making across the range of modalities. Thus, “representations” are the ways to present ideas, which can be discipline-specific or more generally showing visualizations of concepts or thinking possibly using several modalities. Papers in this Special Issue also include a range of settings and contexts for science learning from early childhood to high school students and preservice teachers.

We acknowledge and thank the many reviewers who graciously accepted our invitations to review the submissions for this Special Issue and offered constructive feedback for the authors. We also thank the authors and teams who have critically engaged with the feedback and produced a very interesting set of papers.

In the original Call for Papers in this Special Issue, we invited contributions from science education researchers who were working with social semiotic theory to develop understandings of how learners make meaning with multiple representations. We specifically imagined work from a range of learning contexts and settings and expected that each paper would contribute theoretically to this contemporary area of research focus.

Papers are grouped into three sections. In the first section, papers focus specifically on theory building. Systemic Functional Linguistics/Systemic Functional Theory is foundational to many papers in this Special Issue, even as the research reported offers insight to methodological issues and analytic challenges in different contexts for learning. In the second section, papers focus on student-generated representations and how the processes of production help us to understand meaning-making with representations. The third section moves into classrooms to consider how teachers orchestrate the use of multiple representations to support student meaning-making.

Building Theory

Kok-Sing Tang and colleagues demonstrate important connections between multiple representations in different modes and genre. They offer helpful advice for why attention to genre is important in teaching and learning activities with multiple representations.

Joe Ferguson offers a way to focus creative and logical thinking through a lens of reasoning about representations as high school students work multimodally with digital simulations and models. His theoretical grounding in Magnani’s (2001) notion of “abductive reasoning” and Peirce’s (1894/1998) “logic as semiotic” offers theoretical tools for why working with multimodal representations is productive for learning.

Also grounded in Peircean semiotics, Prain and Tytler propose “a pragmatist account of transduction” through their work with high school students. Using a Representation Construction Approach to the representational challenge, analysis shows how learners reasoned with representations and sought correspondence or alignment among the various sign systems. The learners constructed models as part of this work as they moved between modes and different meanings to develop understanding.

Student-Generated Multiple Representations

As students generate explanations of science concepts, they work with representations and draw on a range of conventions in communicating meanings. Yeo and Gilbert provide an in-depth analysis of the semiotic work students do as they progress through a series of representations in developing explanations for key concepts in physics. Importantly, Yeo and Gilbert theorize that conceptual shifts toward abstract and relational thinking are mirrored by shifts toward more symbolic and formal types of representations utilized. Importantly, this work provides a context for learners to engage with the underlying scientific theories and relevant representational forms, demonstrating how their reasoning becomes more sophisticated with the more advanced forms of representation.

Pham and Tytler developed a “bridging representation” for high school chemistry students that supports their learning about molar concentration and solution chemistry. Using multiple representations across Johnstone’s (1993) levels of interaction (sub-microscopic, symbolic, macroscopic), the authors draw from Peircean semiotics to illustrate how bridging analogies can be analyzed semiotically.

Also studying student-generated explanations, Nielsen et al. present a case study of a preservice teacher who navigated multiple modalities and representational forms to create a “digital explanation” as a teaching resource for children. Analysis using the register variables of field, tenor, and mode shows how the non-linear process of creating the digital product is actually a meaning-making process for the creator.

Bridging student-generated multiple representations and teacher orchestration of multimodalities in the classroom, Wanselin and colleagues developed an analytic framework based in Halliday’s metafunctions that moves beyond textual analysis of teaching resources to include student-generated texts.

Multimodal Orchestration in the Classroom

To conclude the set of papers in this Special Issue, papers in this section look at how teacher actions and choices of representations guide students’ engagement with science content. The teachers utilize a range of semiotic resources in combination with multiple representations to support student meaning-making.

In the early years, Arejung and colleagues compared teacher-introduced representations with student-generated drawings to interrogate how early years children interpret and incorporate the represented science content in their own drawings. Children’s meaning-making comes through the choices teachers make to share particular kinds of semiotic resources and is supported through the teacher’s attention to details of images and meanings but also how representational conventions for scientific drawings are conveyed. Arejung et al. link children’s “emergent disciplinary drawing” to emergent disciplinary literacy as young children learn about specialized practices, including drawing, in different subject areas.

In another early childhood learning context, Kewalramani and Veresov integrate perspectives on child development from Vygotsky and semiotics from Halliday to analyze a play-based creative inquiry where robot toys mediated children’s meaning-making. The authors demonstrate how even very young children can fruitfully engage with scientific concepts when multimodal experiences are appropriately mediated.

Making a new link between multimodalities and variation theory, Preston and colleagues, working with elementary school children, used a “Representation Construction Approach” to develop a pedagogical sequence of activities around electricity concepts. Preston et al. show how the teacher orchestrated meaning-making through multiple modalities including drawing and directed students’ attention to critical features of the concept.

Teacher gesture as part of orchestration with multiple representations is key to work in two other papers. Kevin Tai elaborates the importance of gesture as an important modality in classrooms where English is the language of instruction but many of the students are not native-English speakers. Tai links “translanguaging” in high school science and mathematics classrooms in Hong Kong to learners (and teachers) who are often working in languages other than their own L1 or even L2. Working with text and images in high school biology texts, Ngo and colleagues further illustrate the idea of multimodal orchestration through gesture and language by the teacher and use an SFL framework to show how these are important for student meaning-making in work with scientific diagrams and images.

The collected papers in this set offer state-of-the-art interpretations of social semiotic theory as applied to science learning across a range of contexts and age-ranges. Understanding how learners make sense of representations, and how teachers orchestrate student work with representations, are shared purposes for each of the papers. The different approaches and contexts make for interesting reading and application. As is clear through the set of papers in this Special Issue, multiple representations, multimodalities, and social semiotics are fruitful areas of study in science education research. We invite you to engage with this work through this Special Issue collection.