When working with multimodal texts in education the starting point must be the ways in which contemporary texts are constructed: they are multimodal and composed of various textual elements, something which can be challenging for the learner who is not yet familiar with a specific content area or text type. If teachers guide their students into an understanding of the ways in which texts are designed and how the information structure can be understood, they can harness their students’ content learning. Other central aspects for teachers working with texts in education include developing a greater understanding of student-created multimodal texts, and developing assessment criteria that do justice to such texts.

The digitized media landscape affects society in several ways. In school contexts, there are numerous examples of how contemporary text and media worlds affect the ways in which educational settings are designed. One such example is the flipped classrooms where teachers provide their students with video-recorded outlines or lectures, or provide other learning resources digitally for the students to prepare before actually meeting in the classroom. During lessons in the physical classroom, time and effort can then be devoted to discussions, experiments or other types of activities. The access to digital resources affects the ways in which both teachers and students use their time, individually or in groups. However, regardless of the use of paper-based (textbooks and other types of texts) or digital media, students might need guidance in how to approach the texts.

1 Texts in and Outside of Schools

In contrast to the situation that prevailed when the authors of this book went to school, when there were still school subjects in which texts were not used at all, or to a very little extent, or where the subject content was mainly based on practical training, or trying out different techniques (such as physical education or art), theoretical discussions have become an integrated aspect of most subjects (to our knowledge, this is the situation not only in the Scandinavian countries, where we have our main experience). However, “theory” does not necessarily have to be equivalent to “written texts”, but even so, written texts have become increasingly important in many different subjects, as is evident in the curricula in many countries. This change is also evident outside of school contexts; in increasingly more workplaces employees are expected to use writing in quite complex ways, even in contexts that we might perceive as mainly manual (e.g. Karlsson 2009; Nikolaidou 2014). Our spare time, too, is filled with various text activities, from shopping lists and the planning of leisure activities to text messaging, Facebooking or Twittering (e.g. Barton and Hamilton 1998; Davies 2012). Thus, we live today in a text-based society with partly new types of text cultures.

To encounter a new disciplinary content in school always involves new ways of communicating. A lot has been written about the ways in which texts are structured linguistically in different content areas (e.g. Schleppegrell 2004; Martin and Rose 2008). Alongside this growing awareness, researchers have stressed the importance of working with language in conjunction with content in all school subjects. Such a way of dealing with disciplinary language (see Shanahan and Shanahan 2012, for a discussion on “subject literacy” vs. “disciplinary literacy”) has been a central aspect in genre-based pedagogies, where one concern has been to find ways to support students in ways that promote their possibilities to become part of the discourse of different subject areas and the ways in which ideas are communicated within them (Christie and Derewianka 2008; Fang and Schleppegrell 2004). Regarding the use of other meaning-making resources than verbal language (written or spoken), however, there are obvious differences between disciplines too. Not only terminology or the ways in which the verbal text is structured differ between, for instance, a book spread in history and one in chemistry. Visual resources such as images, diagrams, and tables are just as important concerning textual choices. Therefore, learning a new subject and becoming part of the discourse in that subject involves dealing with many different meaning-making resources. Furthermore, it is essential to be conscious about when and how one meaning-making resource is the most efficient and appropriate to use (cf. affordance, below).

2 Text and Literacy—Changing Concepts

Parallel to the extension of the text concept, literacy has an extended definition today. Within the tradition of New Literacy Studies (NLS) (Barton 2007, Barton and Hamilton 1998) it has long been claimed that reading and writing are not delimited competencies for an individual to develop once and for all. Instead, from the perspective of NLS, encounters of new discursive areas entail new literacy practices. Therefore, during the course of our lives, we will be presented to new literacy challenges again and again. So far, New Literacy Studies have mainly focused on literacy practices in people’s everyday lives (e.g. Barton and Hamilton 1998) or at workplaces (e.g. Karlsson 2009; Nikolaidou 2014). However, the question about literacy practices in different contexts is as relevant for the school context, and not least in Scandinavian contexts, researchers have been inspired by this way of approaching texts and text use. The question of literacy practices within different fields of knowledge is also relevant from the multimodal perspective adopted in this book.

3 Towards a Digital Text World

The digitized world of today has brought new possibilities regarding text design and text production (e.g. Selander 2015; Sheridan and Rowsell 2010). To put it simply, anyone who uses digital media in their text production also becomes a text designer. A slightly more complex perspective on digitization is to view digitized texts as texts that involve new possibilities, such as integrating links to other texts or web pages, to use color for various purposes, to make animations, and to integrate sound or film clips in the text, and so on.

Digitization has also enabled new forms of mobile learning. One example is the use of mobile phones to perform mathematical calculations or to analyze water quality during excursions in nature, while simultaneously being able to communicate with students in other countries working with similar themes (Eliasson 2013; Nouri 2011). Thus, by using digital techniques, online communication is enabled, where other students can be invited to comment on the types of texts in use. Therefore, both texts and the context of creating texts are extended in quite radical ways.

Traditional paper-based texts—in which sentences are connected to paragraphs and paragraphs are connected to a whole—can be seen as more or less linear and with clear boundaries: we know where the text begins and where it ends. A digitally produced text can be linear in the same fashion, but the medium enables other possibilities that open up for a more flexible reading order. For the reader, therefore, the reading process differs from traditional texts. In a digital text, the question of beginning or ending is not self-evident. For example, should linked texts—possibly involving yet other links—be considered part of the text or not? A Facebook page is constantly under revision (at least with an active user), and the owner as well as other participants can add features such as text, images, or links.

Not the least among young people, applications such as fan fiction are popular, that is, websites where fans can write stories based on existing fictional worlds, such as popular novels, computer games, or television series, and where participants encourage each other and comment on each other’s texts. In such communities, anyone who is able to create a text with a style similar to the original author can be expected to be particularly successful. In Chap. 5, “Cultivating Students’ Text Creation”, we will discuss digitally created texts in relation to text compilation and the use of references to sources (that is, “who owns the text?”). In school contexts, compilation texts are often regarded as problematic. In relation to, say, fan fiction, however, the question of ownership of texts is a difficult one, and maybe it is actually an irrelevant question altogether? This is how WideEyedDreamer01 at fanfiction.net comments on his/her Harry Potter text: “Disclaimer: I don't own the Harry Potter series, it all belongs to JK Rowling. Enjoy and Review!” (fanfiction.net 2013).

When students work with digital media, one might expect a spontaneous use of multimodal possibilities, and the integration of various types of visual resources. Yet this is not always the case. A study from chemistry classrooms revealed that the digital possibilities were mainly used to create artistic headings (Danielsson 2011). In another project, students in a physics classroom complained when they were asked to use their tablets when drawing, commenting that it is easier to use pen and paper (Danielsson manuscript). In the chemistry classroom, students were relatively unaccustomed to using digital resources for laboratory reports. Furthermore, in many ways it is still easier to make drawings by hand.

Digital versions of textbooks are often examples of texts where the digital version is close to the printed one and where the dynamic possibilities are utilized only to a little extent. In previous textbook analyses, we have noted that textbook publishers quite frequently launch a digital version in parallel to a paper-based version of the same book (Danielsson and Selander 2014). Sometimes, but not always, the digital version includes the possibility for users to listen to the text by clicking on sections. Another option is to provide the user with the possibility to navigate between sections through links in indexes, or to include search functions, or the possibility to make digital notes in the text. In many of the texts we have analyzed, illustrations on the book spreads were identical between the paper-based and the digital version. Perhaps one might have expected greater differences between the versions given the potentials for digital media. However, it is likely that in the near future we will see rapid development in this area.

Today it is common for students to use the Internet to search for information, at least in schools where students are given access to computers or tablets. Of course, there are great differences as to the economic and practical conditions both between and within countries. For younger kids, there is a plethora of educational learning sites with links to other digital resources. In Part II of the book, we comment on one such site, produced by National Geographic.

An interesting Swedish study was performed by Ulrika Nemeth (2011), who investigated the use of web pages among upper secondary students during school lessons. In the classroom there were discussions about the reliability of sources, for instance in relation to the producer of the page. What was particularly interesting from a multimodal perspective, however, was that when students were searching the Internet for information related to assessments, the students themselves discussed reliability from other perspectives, such as layout. From these discussions it was obvious that the students considered pages with an inconspicuous design to be more reliable than colorful pages, or pages with many animations.

The use of film clips from YouTube, produced for formal learning, is another example of digital resources utilized for pedagogical reasons. What is striking in this regard is the variety of presentational styles in such clips. There are quite a few examples where, for instance, PowerPoint notes, or video-recorded blackboard notes, are commented on through a relatively traditional spoken lecture. These types of recorded presentations can be seen as parallel to digital versions of printed textbooks, where it is mainly the media of presentation that has been altered, but not really the form of presentation. In, for example, the area of natural sciences, on the other hand, the technical possibilities of digital media have been used to produce relatively advanced animations (for instance regarding the relation between the sun and the earth), video-recorded experiments, or video clips like “Chemical Dance Party”, with students acting as different chemical substances to illustrate the substances’ tendencies to react with other substances.Footnote 1

The changes of the conditions for texts and text-work in educational settings also include the use of games and simulations (Arnseth et al. 2019; Brooks et al. 2021; Gallagher 2015; Mørch and Thomassen 2016). One example is the Swedish game Minecraft, which we comment on in more detail in Chap. 13, “Educational Games and On-line Resources”.

When using games for learning purposes, it is important to make the learning goals and learning objects explicit to the students. Also, the teacher can combine the use of games with other tasks, “outside” of the actual game to benefit from the joint engagement in the classroom.

In an international developmental project, researchers and teachers from Australia, Ireland, and Sweden collaborated in developing a game for collaborative problem solving and on the assessment and teaching of “21st Century Skills” (Nouri et al. 2017). In this game, students worked in pairs to solve a joint problem. The idea was that the two students had access to different information about the same problem and they needed to collaborate to get access to all information. Also, they needed to collaborate to find out what the problem actually was about and what information was relevant for solving the problem. To adapt the games to different educational contexts, researchers and teachers in the different countries adapted the games to be in line with the curricula and the specific learning goals in each country.

The teachers noticed some specific problems related to working with the cases and the assessment of the cases: one was that information about the group work had to be very precise in order for the students to be able to handle the cases; the other problem was that it was not evident which school subject “owned” a problem like “the pollution of the Baltic Sea”, and was thus responsible for the assessment: was it Social Sciences, Chemistry, Physics, or History, or a combination of these subjects? In sum, the development of these games point towards interesting new possibilities: (i) to develop small-scale and locally adapted educational games, (ii) to include game-oriented elements in the learning activities, (iii) to use the opportunity to work collaboratively between subjects.

Hence, just as there is no typical paper-based text, there is no typical digital text. In educational contexts, the same challenges are connected to digital texts as for traditional, paper-based texts. Therefore, students might need different kinds of support when choosing and making meaning from digital texts. Thus, it is important for teachers always to consider the function of the various ways of (re-)presenting content, and how different choices, such as animations of ionic bond dancing at a “chemical party”, might be challenging in students’ meaning-making process.

As mentioned, in educational contexts, the flipped classroom, or flip teaching, is growing. Using such ways of designing teaching is one way of employing digital media to be able to use time in the classroom in partly new ways. Often what students have traditionally been provided with in the classroom—such as a theoretical background to a content area—is now given through digital resources outside the classroom, and the students get access to these teaching resources through, for example, a blog. An example of flip teaching is when students are asked to view recorded lectures or PowerPoint presentations produced by the teacher, or freely accessible on the Internet. In that case, students can use the teaching resource in ways that fit their respective interests or learning conditions, and the classroom time can be used for activities like problem solving, group discussions, or hands-on activities, such as experimental work. When organizing learning activities in such ways, the time in the classroom can be used to focus on implementing the content that has been provided through digital resources, and for activities where the teacher interacts with the students in various ways.

The underlying idea behind the flipped classroom is for teachers and students to use classroom time for communicative activities or investigations of different types, and for the teacher to support the students in various ways. One benefit—apart from being able to utilize joint classroom time for communicative activities—is said to be that students often find classroom activities more meaningful. However, such a setup requires that students both have access to and are able to handle digital media, and also that they are able to take responsibility for their own learning outside the classroom.

4 Summary

In this section we have mentioned a number of changes regarding text practices in and outside of school. All of them affect curriculum documents and ways of formulating requirements regarding what counts as knowledge and how to assess it.

The digitized society has led to new possibilities and challenges regarding text design and text production. Everyone who uses digital media has also become a text designer. Furthermore, digital texts provide us with new affordances, such as the possibility to incorporate links and to add audio- and film-clips in texts. The Internet has also led to a new culture of sharing. One effect of such a sharing culture is that the limits of a certain text are no longer evident, and it is easier to integrate other texts in new texts, something which has consequences for education (Selander 2017). The digital development has also created new forms of teaching, such as the flipped classroom.