Promoting learning transfer in science through a complexity approach and computational modeling

Abstract

This article concerns the synergy between science learning, understanding complexity, and computational thinking (CT), and their impact on near and far learning transfer. The potential relationship between computer-based model construction and knowledge transfer has yet to be explored. We studied middle school students who modeled systemic phenomena using the Much.Matter.in.Motion (MMM) platform. A distinct innovation of this work is the complexity-based visual epistemic structure underpinning the Much.Matter.in.Motion (MMM) platform, which guided students' modeling of complex systems. This epistemic structure suggests that a complex system can be described and modeled by defining entities and assigning them (1) properties, (2) actions, and (3) interactions with each other and with their environment. In this study, we investigated students’ conceptual understanding of science, systems understanding, and CT. We also explored whether the complexity-based structure is transferable across different domains. The study employs a quasi-experimental, pretest-intervention-posttest-control comparison-group design, with 26 seventh-grade students in an experimental group, and 24 in a comparison group. Findings reveal that students who constructed computational models significantly improved their science conceptual knowledge, systems understanding, and CT. They also showed relatively high degrees of transfer—both near and far—with a medium effect size for the far transfer of learning. For the far-transfer items, their explanations included entities’ properties and interactions at the micro level. Finally, we found that learning CT and learning how to think complexly contribute independently to learning transfer, and that conceptual understanding in science impacts transfer only through the micro-level behaviors of entities in the system. A central theoretical contribution of this work is to offer a method for promoting far transfer. This method suggests using visual epistemic scaffolds of the general thinking processes we would like to support, as shown in the complexity-based structure on the MMM interface, and incorporating these visual structures into the core problem-solving activities.

Introduction

In educational settings, one challenging task for educators is how to better design classroom learning activities that encourage students to transfer their learning across domains. Bransford et al., (2000) defined learning transfer as the ability to use the knowledge learned in one context within a new context. The literature has distinguished between two types of transfer: near transfer, which occurs between two similar contexts, and far transfer, which occurs between two superficially dissimilar but abstractly related contexts (Barnett & Ceci, 2002; Day & Goldstone, 2012; Gentner, 1983; Hummel & Holyoak, 2003; Klahr & Chen, 2011). Despite the centrality of transfer as a goal, the difficulties in achieving it have been documented by education researchers for many years (Perkins & Salomon, 1992; Salomon & Perkins, 1989; Thorndike, 1906).

One explanation for these difficulties is that learning transfer requires a deep understanding of the source problem before transferring to the target domain, which in educational settings does not usually happen within short durations because the source domain has just been learned (Chi & VanLehn, 2012; Lobato, 2006; Marton, 2006). One practice that has shown increasing promise in promoting such deep conceptual understanding is engaging students in the active construction of models (Dickes et al., 2016; Gravemeijer et al., 2000; Lehrer & Schauble, 2006). In this study, we explore the potential of this practice to promote transfer by means of the Much.Matter.in.Motion (MMM) platform (Levy, Saba, & Hel-Or, 2019)—a block-based modeling platform that enables middle-school students to learn about systems in chemistry and physics by constructing a wide range of computational models in these domains. We compare the learning outcomes of two groups of seventh grade students who learned about the topic of gases—one via the regular chemistry curriculum, and the other by using the MMM platform to construct models. In addition to conceptual learning, the platform is also designed to promote systems understanding, and the development of computational thinking (CT) competences.

“Systems and system models” have been defined by the Next Generation Science Standards (NGSS, Lead States, 2013) as a key, cross-cutting concept in STEM education. Complex systems are composed of many elements, which are self-organized in coherent, global patterns, interacting dynamically both among themselves and with their environment (Bar-Yam, 2003; Epstein & Axtell, 1996; Holland, 1998). “Systems understanding” therefore entails an understanding of how the system’s elements interact at the micro-level (micro-level understanding), how its phenomena can be understood at the macro-level (macro-level understanding), and how a higher-order or collective behavior may emerge from these interactions, linking micro and macro level understandings.

CT has also been identified as a core practice by the NGSS (2013). It encompasses the ability to solve problems, design systems, and understand human behavior in ways that are related to the ideas behind computation. It includes decomposing difficult problems into smaller and easier ones that can be solved, the use of recursive thinking, pattern finding, and abstraction (Wing, 2006). Several studies in educational STEM emphasize the fact that CT can be practiced across content-domains in STEM. These studies address the impact of integrating CT into a learning process that focuses on enhancing both CT and conceptual understanding through computational modeling of complex systems (Basu et al., 2014; Blikstein & Wilensky, 2009; diSessa, 2000; Guzdial, 1995; Hambrusch et al., 2009; Kaput, 1994; Wilensky & Resnick, 1999; Zhang & Biswas, 2019).

Similar to other block-based modeling platforms (such as CTSiM and EvoBuild), MMM is based on the Agent-Based modeling (ABM) approach to complex systems, which focuses on identifying a system’s entities and defining their behaviors and interactions (Bar-Yam, 2003). One important innovation of the platform’s design is that the visual representation of its coding interface is shaped by the “complexity-based structure” conceptual framework which is based on the ABM approach (Fig. 1). The complexity-based structure framework suggests that a complex system can be described and modeled by defining the entities in a given system and assigning them (a) properties, (b) actions, and (c) interactions with each other and with their environment. For example, a gas inside a container can be modeled by defining the entities of the system, namely gas particles, assigning them properties (randomly set headings, initial speed), actions (moving in straight lines), and interactions (when particles meet with another particle they collide with each other, changing their heading and speed, or when particles meet with the boundary of the container they bounce, changing only their heading). Upon running the simulation, one can then observe several macro-level phenomena emerging from these micro-level interactions, such as how pressure changes when particles are added, or how their speeds are changed.

Fig. 1

The Much.Matter.in.Motion (MMM) interface, Levy, Saba & Hel-Or (2019). Right (green shape): “Complexity-based structure”

The study described in this paper was designed to address several distinct but interrelated goals. The first goal was to determine the impact of using the MMM modeling platform upon students’ conceptual knowledge, systems understanding, CT and—most particularly—knowledge transfer. The contribution of ABM to conceptual knowledge and systems understanding has already been shown in various studies (Samon & Levy, 2017; Brady, Holbert, Soylu, Novak, & Wilensky, 2015; Dickes, et al., 2016), and the contribution of this particular platform to these two factors has already been explored in-depth by us, in a previous paper (Saba, Hel-Or & Levy, 2021). However, this platform’s contributions to the development of CT, and the potential relationship between computer-based model construction and knowledge transfer, have yet to be explored. (It is important to mention that the learning design in this study did not aim to “teach for transfer”.) We therefore conducted a pre and post analysis of students’ responses to a series of dedicated questionnaires on conceptual knowledge, systems understanding, CT and, knowledge transfer, and compared the scores of an experimental and comparison groups that studied with the normative instruction.

Our next goal was to look more closely at the potential contribution of the MMM platform’s complexity-based structure to the possibility of knowledge transfer. To achieve this, we conducted a qualitative analysis of the students’ responses to questions designed to assess both near and far transfer, to see whether the MMM platform’s use of the complexity-based structure as its visual scaffolding was reflected in the form and content of the students’ explanations. Visual scaffolding refers to representation of conceptual objects in a visual format (Rha, 2007). In our study, the visual scaffolding uses the complexity-based structure to model the components and behaviors of complex systems. The third and final goal of the study was to trace the ways in which the three factors—conceptual knowledge, systems understanding and CT—may affect one another, and whether each of these three directly or indirectly impacts knowledge transfer. This goal was addressed by means of a “path analysis” (Ahn, 2002) of the students’ scores to determine the mutual relationships and influences of these four factors.

Literature review

Constructing models as a means of learning about complex systems

Constructing models is a core activity in this study. Researchers that focus on modeling in science education have defined models as representations of a phenomenon initially created for a specific purpose (Gilbert et al., 2000; Houseal, 2016). Model construction simplifies the phenomenon of interest based on the future use or the goal of the model, and it can serve as an explanatory tool (Gobert & Buckley, 2000). Constructing computational models allows students to conduct several changes iteratively and to progressively refine their model in order to explore the phenomena under investigation.

Studies have addressed the significant advantages of learning via the construction of computational models of complex systems to the improvement of students’ conceptual learning and systems understanding (Saba, Hel-Or & Levy, 2021; Wilkerson-Jerde et al., 2015). Furthermore, studies such as Wagh and Wilensky (2018) have addressed the significant affordance of engaging in constructing computational models, using the EvoBuild modeling tool, over exploring prebuilt models. They found that students who constructed computational models expressed greater learning of evolutionary mechanisms than did students who explored prebuilt models.

Several approaches for modeling complex systems in science education have been introduced in the literature, including the Structure, Behaviors, and Function (SBF) approach (Assaraf et al., 2013; Eilam & Poyas, 2010; Goel et al., 2009; Liu & Hmelo-Silver, 2009; Simon, 1969), the System Dynamics (SD) approach (Forrester, 1968), and Agent-Based Modeling (ABM; Bar-Yam, 2003). In this study, we adopted the latter approach, namely the ABM of complex systems. The ABM approach represents systems through their participating entities (i.e., agents), assigning them behaviors and interactions. Running the simulation shows how these entities act and interact, a process that results with an emergent collective pattern in a bottom-up way.

We selected this approach because of its generativity in science as well as its ability to help students relate micro and macro levels (Levy & Wilensky, 2009; Wilensky & Resnick, 1999). For example, an ant convoy is an emergent phenomenon that results from micro-level interactions between single ants, food sources, and pheromones. When an ant finds food, it releases pheromones that evaporate. Other ants look for pheromones, heading for the strongest scent. The actions of many individual ants at the micro level can reach a critical mass at the macro-level, resulting in a path of pooled scent, which causes the emergent phenomenon of an ant convoy marching.

Several studies have demonstrated the significant advantage of the ABM approach to complex systems in promoting students’ systems understanding and conceptual learning (Samon & Levy, 2017; Brady, Holbert, Soylu, Novak, & Wilensky, 2015; Dickes, Sengupta, Farris, & Basu, 2016; Holbert & Wilensky, 2014; Sengupta & Wilensky, 2009; van Mil, Boerwinkel, & Waarlo, 2013). The potential contribution of using computer-based simulations to knowledge transfer has also been addressed in a variety of studies (Falloon, 2020; Goldstone & Sakamoto, 2003; Goldstone & Wilensky, 2008; Hustad et al., 2019). Using computer simulations allows students to interact with the simulation and interpret objects and their interactions. Especially when these interpretations are idealized, the set of relations in one setting can be used in dissimilar situations, thus promoting the transfer of systems understanding (e.g. micro-level, macro-level, emergence etc.) across domains (Goldstone & Sakamoto, 2003; Goldstone & Wilensky, 2008). However, some of these studies, such as Falloon (2020) and Hustad et al. (2019), do not rely on a complex systems approach, which is fundamental in this study. Other studies (Goldstone & Sakamoto, 2003; Goldstone & Wilensky, 2008), use the complex systems approach, but rely on exploring pre-built computational models and do not enable students to construct these models themselves, which is also a core element in this study.

The synergy between CT and STEM through model construction

Recent studies, which have explored the advantages of using computational modeling tools in the teaching of complex systems, have also highlighted the synergy between CT and STEM (e.g., CTSiM: Basu et al., 2014; DeltaTick: Wilkerson-Jerde et al., 2015; EvoBuild: Wagh et al., 2018).

Studies have shown that applying CT in STEM domains through modeling enhances students’ learning (Basu et al., 2016; García-Peñalvo, Reimann, Tuul, Rees, & Jormanainen, 2016; Gadanidis et al., 2016; Jaipal-Jamani & Angeli, 2017; Pei et al., 2018; Zhang & Biswas, 2019). These studies show that programming for computational modeling can serve as an effective vehicle for learning challenging science and math concepts. For example, CT2STEM (Hutchins et al., 2020) is a collaborative, computational learning environment. It combines visual model construction with a domain-specific modeling language to scaffold learning of high school physics using a computational modeling approach. The result of the study indicated that students who worked with C2STEM developed a better understanding of concepts and practices in physics and CT than students who learned through a traditional curriculum.

One main benefit of embedding CT into STEM classrooms is developing a mutual relationship between math, science, and CT, in a way that aligns with recent scientific practice (Weintrop et al., 2016). It is particularly beneficial when the learning of STEM is based on the complex systems approach. Berland and Wilensky (2015) found that programming many robots in a single environment develops both understanding of complex systems and CT. They explained that these types of understanding are mutually reinforcing because of the agent and aggregate perspectives the students adopt. Another advantage of integrating CT into STEM is that it allows students to intuitively formalize scientific phenomena based on computational mechanisms and principles, rather than using abstract mathematical principles (Redish &Wilson, 1993; Sengupta & Wilensky, 2011; Wilensky & Reisman, 2006).

Having established the idea that CT can be a useful tool for learning STEM, however, we must then ask—how should we help our students acquire CT? The conventional view today is that basic concepts in computer science and programming are an essential component of CT, and that these concepts should be introduced to students from a young age (Brennan & Resnick, 2012; Cooper et al., 2000; Grover & Pea, 2013; Weintrop & Wilensky, 2015; Werner, Denner, Campe, & Kawamoto, 2012). Accordingly, teaching CT is often based on having students learn programming skills. For example, Scratch (Resnick et al., 2009) and Alice (Cooper et al., 2000) are popular programming languages that enable students to learn CT competencies through the construction of games and simulations.

Concerns have been raised, however, that direct teaching of CT through programming alone might reduce students’ interest in learning it (Mooney et al., 2014). In addition, the National Research Council (2011) recommended that CT and programming be integrated within the K–12 science curricula in more significant ways. This integration requires a shift from teaching CT, programming, and modeling as separate topics to designing science domain learning environments that focus on interweaving these competences in learning (Sengupta, et al., 2013). The MMM (Levy, Saba & Hel-Or, 2019) platform used in this study is designed with such an integration in mind—interweaving CT development with the teaching of STEM content through the construction of computational models.

The transfer of learning: definitions and methods to increase transfer

Within educational systems, educators usually strive to effect positive changes not only to the specific topic of learning but also beyond it. Underlying this effort is an assumption that different kinds of knowledge may share common structures. Thus, educators aim to design learning experiences that enable students to use such common structures of knowledge across courses, school years and in their workplaces (Bransford & Schwartz, 1999). These learning experiences may rely on educating students broadly rather than training them to execute certain tasks (Broudy, 1977). In addition, assessing learning transfer can help educators to measure the quality of students’ learning experiences. This is because, when assessing both learning and transfer some kinds of learning experiences might produce effective memory but poor transfer; however, others may achieve effective memory and positive transfer (Bransford et al., 2000).

The failure to transfer learning from one context to another is a well-known problem in education research. This is particularly true when learning must be transferred between different contexts that have similar deep structure yet are dissimilar superficially, namely (far transfer). Some researchers have gone so far as to deny the existence of far transfer altogether (Barnett & Ceci, 2002; Denning, 2017). Other researchers have found a very limited degree of far transfer, such as Sala and Gobet (2017), who studied the topic in the domains of chess instruction and music education. Chi and VanLehn (2012) named this problem as the “failure-to-transfer phenomenon.”

Classically, transfer of learning is described by Lave (1988) as two processes in a “two problem transfer paradigm,” namely the first process of initial learning followed by the second process of applying the learned knowledge. One of the basic distinctions is between surface and structural similarity as a basis for learning transfer. A problem’s “surface feature” refers to the perceived concepts, or entities, that have an explicit description in that problem, so that transfer by surface similarity is based on reminding cues and knowledge application. “Deep structure,” on the other hand, indicates the procedures for solving a problem that often cannot be directly recognized. Transfer, researchers have noted, often fails when the two problems have dissimilar surface features but a similar deep structure (Chi &VanLehn, 2012; Gick & Holyoak, 1983).

Much work has been done on the structural similarity between contexts and on the process by which far transfer could take place. Transfer by structural similarity takes place by mapping between the source and target situations (Day & Goldstone, 2012; Gentner & Hoyos, 2017; Hummel & Holyoak, 2003): by matching between the relations within systems in two dissimilar contexts, which involve different objects and features (Day & Goldstone, 2012; Gentner, 1983; Hummel & Holyoak, 2003; Klahr & Chen, 2011; Malkiewich & Chase, 2019). Malkiewich and Chase (2019) go one step further and emphasize two constructs related to transfer by structural similarity. One construct, which aligns with most previous studies (e.g. Chi &VanLehn, 2012; Day & Goldstone, 2012), is noticing deep structure as an important factor for learning transfer. It is defined by recognizing and interpreting certain information in the problem. The second construct is focusing, which is defined as “choosing to engage with noticed information over time and determining whether it is important for task success” (Malkiewich & Chase, 2019, p, 1477). These researchers found that students who had the ability to focus on deep structure when engaging in engineering tasks succeeded in transferring science concepts to non-engineering problems.

In our study, we build upon Malkiewich and Chase’s (2019) work. The MMM modeling tool, we hypothesize, assists students in noticing the deep structure of complex systems through the explicit visualization of the complexity-based structure reflected in the MMM interface. This visual scaffolding, we suggest, also encourages students to focus on the core aspects of a phenomenon when solving a transfer problem, in the case of chemical and physical systems modeled with MMM, the micro-level interactions. It does this by allowing students to draw the macro-level objects in the system, select and place the micro-level entities in the model, and then code the rules for their behavior.

Barnett and Ceci (2002) present a framework for far transfer by offering a taxonomy that is characterized by two main categories: the content—“what is transferred” and the context—“when and where it is transferred from and to”. In the category of context, five dimensions are presented: knowledge domain, physical context, temporal context, functional context, social context, and modality (see also Fig. 1, P. 621, in: Barnett & Ceci, 2002). Each dimension can be rated along five gradations ranging from near to far transfer. In our study, we follow the taxonomy presented by Barnett and Ceci (2002) to more finely examine the conditions for learning transfer, which will be discussed in more detail in Sect. “Data collection instruments”. We used two key dimensions of the taxonomy—“Knowledge domain” (which can refer to broad definitions like “science” and “art”, and to their sub-categories, such as “biology,” “botany,” “chemistry”), and “Modality” (which refers to the problem’s format, e.g., written, multiple choice, oral etc.)—to characterize which tasks could serve for testing near transfer and which task could be considered for testing far transfer.

Methods to increase transfer are based on the above distinction between the process of learning the source problem and the process of applying it to the target problem. One principle requires deep initial learning of the source problem to enable successful transfer to the target problem (Chi & VanLehn, 2012; Lobato, 2006; Marton, 2006;). One method based on this principle of deepening the initial learning is to involve students in identifying the deep principles at each stage of solving the problem (Goldstone & Sakamoto, 2003; VanLehn & Chi, 2012). Another method, which highlights the second process, involves ways by which instruction could explicitly highlight how these principles can be applied in other domains (Fuchs et al., 2003; Rosholm et al., 2017; Salomon & Perkins, 1989; Terwel et al., 2009;). Fuchs et al. (2003) found that instruction that combined explicit teaching for transfer with direct problem-solving instruction was the most effective for both near and far transfer. Catrambone and Holyoak (1989) showed that providing students explicit hints that relate how the previously learned situation contributes to understanding the new situation can help promote learning transfer. However, caution is needed, as a balance between providing enough guidance to enhance learning efficacy and affording too much guidance that interrupts knowledge construction needs to be established (Margulieux & Catrambone, 2019).

Goals, hypotheses and research questions

This study was designed to address several, interrelated goals. First, it sought to examine the impact of using a computer-based tool, namely the MMM platform, to construct computational models of complex systems, not only upon students’ conceptual knowledge and systems understanding, but also upon two additional significant factors—their CT and knowledge transfer. Its first research question was therefore:

• RQ1: To what extent does constructing models of complex systems on the topic of gases with MMM contribute to students’ conceptual learning, systems understanding, CT, and (near and far) knowledge transfer, compared with normative instruction of the subject?

  Our next goal was to determine, if we found that using the MMM did in fact contribute to knowledge transfer, how the visual scaffolding of the modeling tool’s complexity-based structure may have played a part in facilitating that transfer. We therefore asked:

• RQ2: Which elements of the complexity-based structure (namely, Properties, Actions, and Interactions and their related variables, such as size, speed, heading etc.) do the students in the experimental group use when answering a question that requires near transfer, compared with the comparison group? Which elements do they use when answering a question that requires far transfer?

  The third and final goal of the study was to determine the relationships between the four factors addressed in RQ1, and particularly whether each of the first three factors directly or indirectly influences knowledge transfer. We therefore asked:

• RQ3: What are the contributions of conceptual learning, systems understanding, and computational thinking to knowledge transfer when engaging in learning by modeling with the MMM platform?

In this context, we constructed a path diagram of the four variables, which is shown in Fig. 2. It reflects the following hypotheses: (1) conceptual knowledge, systems understanding, and CT each affect students’ transfer of learning; (2) the three variables, conceptual knowledge, systems understanding, and CT are related in the path analysis: conceptual knowledge and systems understanding each contribute to cultivating CT; conceptual knowledge and CT each contribute to enhancing systems understanding; CT and systems understanding each contribute to cultivating conceptual knowledge.

Fig. 2

A hypothesis path model. CT Computational thinking

The MMM platform

The block-based MMM platform (Levy, Saba & Hel-Or, 2019) is a universal block-based modeling tool for complex systems in chemistry and physics (Fig. 3). It was developed based on a previous NetLogo version of MMM (Levy, Saba & Hel-Or, 2018) and is described in depth in Saba, Hel-Or & Levy (2021). MMM enables students to construct a wide range of computational models of chemical and physical systems based on a complex systems approach and using the visual scaffolding of the complexity-based visual structure in the MMM interface.

The left part in Fig. 3 shows one component of the environment, a model created with NetLogo (Levy, Saba & Hel-Or, 2018; Wilensky, 1999) the right side shows the block-based coding component, which was made with the Blockly open-source library (https://developers.google.com/blockly). Block-based coding provides easy access to programming which, combined with the visual scaffolding in the modeling interface, allows students to construct models with minimal guidance.

The visual representation of the models is displayed in the central window (shown as a black window in Fig. 3) and includes micro-level entities visualized as balls and macro-level boundaries represented as lines and environmental fields represented as arrows.

Fig. 3

The Much.Matter.in.Motion (MMM) interface, Levy, Saba & Hel- Or (2019). Left: NetLogo MMM; right: block-based coding

Figure 3 illustrates a model that was constructed to represent the process of inflating a bicycle tire. The construction of a new model is divided into three steps, which can be iteratively revised and improved:

• (1) Walls within the model (in this case the bicycle tire, which is represented by a blue circle) are painted in by hand using the buttons on the left side of the interface. Using a paintbrush, students can draw shapes to represent containers, electric wires, slopes, and more.

• (2) The right part of the interface is the construction area. It includes a green shape, the complexity-based structure, which has three cavities into which blocks can be dropped. The students drag and drop blocks from a bank with three categories, each in a different color—Properties, Actions, Interactions—into their accorded place inside the green shape. At this stage, the student defines the model’s entities at the micro-level (such as particles, molecules, electrons, planets) and assigns them properties, such as size, speed, and mass (this is done by dragging and dropping “Properties” blocks). All the entities in MMM are represented as balls; in Fig. 3, the red balls represent gas particles inside the bicycle tire. When there is more than one type of entity, each type is considered a group. Each group has its own computational structure, such as two different types of gas, or atoms and electrons in a conductor.

The entities have independent and dependent actions. Independent actions are not affected by the environment. For example, gas particles move in a straight line. However, dependent actions, namely interactions with the environment and other entities, can produce changes in the entities' behavior a, such as changing speed and direction through colliding with another entity, or bouncing off a wall. Dependent and independent actions are assigned by dragging and dropping “Actions” and “Interactions” blocks respectively.

• (C) After completing the micro-level construction, balls are mouse-clicked into place (e.g., the red balls are inserted inside the blue cycle), using the control buttons (Fig. 3, activating the “place-balls” button) on the left side of MMM.

Pressing the “Play” button allows students to dynamically observe the phenomena under investigation; following this observation, it is easy to iteratively manipulate the constructed model.

Over the past ten years, multiple studies have used a variety of modeling tools as a means of enhancing both CT and conceptual understanding of science through the lens of complex systems. MMM, however, differs from these in four key ways: (a) MMM is not restricted to a specific science-content, but enables the construction of wide range of phenomena in chemistry and physics; (b) it affords engagement in scientific modelling through a combination of drawing and model construction; (c) it employs minimal mathematical representations, such as graphs, to support students’ careful observation of the intricacies of the visual model of the complex system itself, rather than its mathematical representations; (d) the core affordance, which is the focus of this study, is the general complexity-based structure that underlies the MMM interface (Saba, Hel-Or & Levy, 2021), which can be transferred across topics and even domains.

Method

Research approach and design

The study was conducted with a quasi-experimental pretest-intervention-posttest and a comparison-group design. It used a mixed-methods approach, combining quantitative and qualitative data analysis (Creswell, 2012). Quantitative analysis was used to compare the experimental and comparison group students’ conceptual learning, CT, systems understanding, and learning transfer. In addition, it was used to explore how promoting students’ conceptual learning, systems understanding, and CT competences may affect the transfer of the complexity-based structure across domains. Qualitative analysis was used to delve into the ways by which the step-by-step processes of learning through constructing models is accompanied by changes in CT practices. In addition, it was used to compare students’ responses to near- and far-transfer items.

Participants

The participants included 79 seventh-grade students (33 girls, 46 boys) from a middle-to-high-socioeconomic-status urban school in Israel. Participants were recruited as four organic, intact classes. Two classes served as the experimental group (n = 38). Two additional classes formed the comparison group (n = 32). Some students did not complete one of the two questionnaires, specifically the posttest, which was conducted during the 2020 COVID-19 pandemic shutdown, reducing the sample to 50 students (24 girls, 26 boys), with 26 pre- and postquestionnaires from the experimental group and 24 from the comparison group. Students from both groups had not learned the topic of gases before participating in the study.

One expert researcher (the first author of this paper), who has experience in teaching, taught the experimental group. The comparison group learned with their science teacher, who holds an undergraduate degree in science education and had been teaching for 18 years.

Research procedure

The learning unit for both the experimental and comparison groups was planned to include six sessions of 1.5 h each, as part of their seventh-grade science class. Because of the COVID-19 pandemic and school shutdown, it included only four sessions of 1.5 h each, without the topic of diffusion. Both groups learned for the same amount of time, and both focused on the topic of gases, including kinetic molecular theory, gas pressure, temperature and the Gas Laws. The experimental group learned the topic of gases using the “Modeling gas behavior with MMM” learning unit. The comparison group learned this topic using a normative approach based on lectures, experiments, discussions, and the use of textbooks. One week before the learning unit and one week after ending it, the students of both groups completed a 25-min questionnaire on conceptual understanding of gases and a 20-min questionnaire on CT. In the posttest, an additional 15-min transfer questionnaire was included. It is important to mention that the learning design of both groups did not “teach for transfer” in the sense described by several researchers as mindful transfer (Salomon & Perkins, 1989), by introducing challenges and examples for extensions of the knowledge beyond the specific topics learned.

The “Modeling gas behavior with MMM” learning unit

In our previous study (Saba, Hel-Or & Levy, 2021), we introduced the “Modeling gas behavior with MMM” learning unit. It is based on the “Gases in Motion” unit designed by Samon and Levy (2017), which is based on chapter 1 of the Connected Chemistry (CC1)¹ learning unit (Levy & Wilensky, 2009). Our unit differs from these in the form by which students were engaged with computer models in the learning environment. The main difference is the practice of constructing models using MMM, rather than exploring pre-built models.

In this study we employed an improved version of the learning unit, based on our previous results and teachers’ feedback (Saba, Hel-Or & Levy, 2021). It included four (1.5 h) lessons in which students used the block-based MMM to construct computational models. Table 1 describes the activities in the learning unit.

Table 1 Activities included in the unit, their duration, and whether the activity involves computational modeling

The instructional sequence consists of three main stages: (1) experience with a physical phenomenon through labs and demonstrations, which were guided by the teacher; (2) students work in pairs to construct models and explore them (i.e., observe the system’s behavior, refine the models based on this observation, posit explanations for the behavior). It is important to note that due to the easy access to programming with MMM and the visual scaffolding of the complexity-based structure in the MMM interface, the teacher’s contribution at this stage is minimal, and mainly focused on helping students with technical problems; (3) classroom discussion and lesson conclusion, guided by the teacher, in which students consolidate their knowledge, present their respective models and explanations, compare them with the physical phenomenon explored in the first stage; after which, students provided their own conclusions, guided by the teacher.

Normative science curriculum

The normative approach to chemistry education mainly relies on the teacher presenting the particulate model as a given theory, demonstrating experiments or involving students in conducting experiments, and exploring phenomena explained by the particulate model. Examples of science experiments include the compression of gas in a syringe and inserting one balloon into a bowl with cold water and another balloon into a bowl with hot water. This approach to learning deals mainly with exploring systems at the macro-level, followed by focusing only on explaining phenomena at the micro-level. Few explanations are based on bridging the macro- and micro-levels of the phenomena, especially in an emergent way that comprises probabilistic aspects and equilibration of the system.

Each lesson is usually divided into the following stages: (1) Introduction and definitions, in which the teachers stands in front of the students, presents the existing theory and basic concepts related to gases and writes them on the classroom board; (2) Teacher lead demonstrations of physical experiments, or inviting students to conduct experiments; (3) Classroom discussion, in which the teacher asks students to explain the results based on the particulate model, either orally or in their notebooks; and (4) Lesson conclusion, which involves the teacher writing lesson conclusions on the classroom board. In our study, the textbook Material Science (Ben Horin, Orad & Welger, 2013), the most common textbook in Israeli schools at the time of this study, was used in the normative lessons (Appendix D includes an example of one lesson).

Data collection instruments

In addition to the “Modeling gas behavior with MMM” learning unit that was developed for this research, we used two identical pre- and posttest questionnaires (Gases questionnaire, CT questionnaire), and two posttest questionnaires (Near-transfer and Far-transfer questionnaires):

• a. Gases questionnaire (Appendix A). The questionnaire tests for conceptual learning and systems understanding regarding science phenomena targeted in the activities. It includes 18 multiple-choice items. These questions have been tested and used in previous research (Levy & Wilensky, 2009; Samon & Levy, 2017). The items involve kinetic molecular theory, and the following concepts related to gas: pressure, temperature, and density. In addition, each item tests for one component of systems understanding: micro level, macro level, and bridging between the two levels of the phenomenon (Appendix A notes, for each item, the system component it tests: micro, macro, micro/macro). For the Gases questionnaire, we used Kuder-Richardson’s Formula 20 to test for internal consistency reliability, ${KR}_{20}$ = 0.86, which indicates for high reliability. The Gases questionnaire was reviewed by two middle school science teachers to ensure clarity and coverage of the concepts and principles taught with the standard learning materials. It was tested, and used in previous research (Saba, Hel-Or & Levy, 2021; Levy & Wilensky, 2009; Samon & Levy, 2017).

• b. CT questionnaire (Appendix B). The questionnaire consists of six items (two multiple-choice, four open-ended) that include pseudo code and prebuilt models. Pseudo code is an informal way of programming, in which verbal descriptions serve as “stand-ins” for computer code. The CT questionnaire was built in-house and reviewed by two middle school science teachers to ensure clarity. Open-ended items 3.1, 3.2 and 3.3 in this questionnaire require students to solve a problem regarding the collective behavior of fish in changing environments.

Near and Far transfer questionnaires were developed based on Barnett and Ceci’s (2002) taxonomy, as described in the literature review. Two dimensions, “knowledge domain” and “modality” were used to define far transfer items versus near transfer items. In terms of knowledge domain, both the near and far transfer items represented a shift into the middle of Barnett & Ceci’s taxonomy—moving within the general domain of “science” from Chemistry in the learning unit (specifically, gases) to biology in the two transfer questions (namely, the behavior of fish and ants). This, on its own, can be considered near transfer because the shift in domains is not extreme. In terms of modality, however, the near transfer items are similar in format to the learning situation encountered by the students in the “Modeling gas behavior with MMM” learning unit, while the far transfer items are not (they require writing explanations, with no support from complexity-based structure). Thus, while the far transfer questionnaire reflects a shift in both knowledge domain and modality, the near transfer questionnaire reduces some of the difficulty in that transition by making the format more similar. Note that, though the content similarity in the near transfer questions is shared by both groups, the modality similarity, which is the combination of modeling, visualizing and code, is true only for the experimental group, which means that the near transfer for the comparison group is “less” near.

Near-transfer questionnaire. Items 3.1, 3.2, and 3.3 of the posttest CT questionnaire were also used as a tool for assessing students’ near transfer. It is important to note that these three items in the questionnaire were analyzed through two lenses—one for CT, examining the code and logic, and once to test for near transfer of knowledge. The near-transfer questionnaire requires transfer of the complexity-based structure employed by the students when learning about gas particles inside a container to the solving of a computational problem related to the collective behavior of fish inside an aquarium. To prevent overlap of the variables CT and knowledge transfer, in the later path analysis, only far transfer was included.

Far-transfer questionnaire (Appendix C). A posttest questionnaire that consists of four open-ended items regarding science phenomena not targeted in the activities. The items require students to solve a problem regarding the collective behavior of ants in changing environments. They were developed in-house, and reviewed by lab members. The transfer items test the transferability of the complexity-based structure when moving from learning about the behavior of gases to explaining the behavior of ants. The far-transfer questionnaire was built in-house and reviewed by two middle school science teachers to ensure clarity. Table 2 illustrates the coding table of students’ responses to the two transfer problems based on the complexity-based structure. 

Table 2 Coding students’ responses to the two transfer problems based on the complexity-based structure. The examples are excerpts from the students’ answers to the questionnaires

Data analysis

Data analysis consisted of four stages.

• 1. Conceptual knowledge and systems understanding: Students’ answers to the questions in the Gases questionnaire were coded as correct (1 point) or incorrect (0 points) for conceptual learning, and a total score was computed and converted to percent out of 100. Similarly, students’ answers to the questions in the Gases questionnaire were coded as correct or incorrect for the systems understanding component (as being at the macro level of the system, at the micro level of the system, or involving micro–macro transitions) and a total score was computed for each component separately and converted to percent out of 100. It is important to note that, as in previous studies (Saba, Hel-Or & Levy, 2021; Samon & Levy, 2017), we utilized the same data from the questionnaire for two distinct analyses: one is the science concepts, where we obtained an overall score. The other is systems understanding, where we disregarded the concepts, but looked at the systemic structure of each item, examining whether it relates to one or more of several systems dimensions, such as micro-level and macro-level behaviors.

Gases questionnaire items were coded by two of the authors. The questionnaire was analyzed with an analysis of variance (ANOVA) test.

For the following three analyses, blind coding was done only by the second coder. The first coder is the first author, who had developed the data categories based on tests. However, the second coder coded students’ responses without any information related to time (pre- post- tests) and condition (experimental vs comparison).

• 2. CT: Students’ answers to the multiple-choice items in the questionnaire (items 1.1 and 1.2) were coded as correct (1 point) or incorrect (0 points). For open ended items, a coding table was developed, using bottom-up coding of the students’ responses, noting the patterns and then selecting the dominant categories, creating definitions and adding examples. It was then reviewed within the lab. Open-ended items in the CT questionnaire were coded by two researchers according to this detailed coding scheme (Appendix B, Table B.1). For items 2, 3.1, and 3.2, the maximum points a student could get were four, and for item 3.2 a correct answer provided students with one point. Thus, the maximum score of this questionnaire was 15 points. The total score was computed and converted to percent out of 100, and the questionnaire was analyzed with an analysis of variance (ANOVA) test. Open items were coded independently by two researchers. Cohen’s (1968) kappa was calculated for each item to test the interrater agreement. Agreement was found to be 0.869 on average. Any remaining disagreements were resolved through discussion.

• 3. Transfer of learning: the near and far transfer problem questionnaires aimed to test students’ use of the complexity-based structure. We sought to determine what the main components are that students use when responding to a near transfer problem, namely the “aquarium problem,” and what main components of the complexity-based structure they use when solving a far transfer problem, namely the “ants problem.”

Near transfer: From the posttest CT questionnaire, the “aquarium problem” (Appendix B), was selected for this purpose. It requires transfer of the complexity-based structure encountered when learning about gas particles inside a container to the solving of a computational problem related to the collective behavior of fish inside an aquarium. A coding table was developed, using bottom-up coding of the students’ responses, noting the patterns and then selecting the dominant categories, creating definitions and adding examples. It was then reviewed within the lab. The maximum total score of this problem was 5 points (Appendix B, Table B.2). The total score for each student was computed and converted to percent out of 100. A Mann–Whitney test was conducted to compare student scores from the experimental group versus the comparison group. Effect size was manually computed by squared Z-value of Mann–Whitney, divided by (N−1). Items in near-transfer questionnaires were coded independently by two researchers. Cohen’s (1968) kappa was found to be 0.891. Any remaining disagreements were resolved through discussion. It is important to note that the near transfer items were used in the pretest and posttest, so that students from both experimental and comparison groups had prior exposure to the problem. Since this specific exposure is similar for the two groups and transfer was based on only the posttest results, only the intervention, which had a similar modality of presentation, is the one that had an impact on the differences between the groups.

Far transfer: the far-transfer questionnaire presents the “ants problem” (Appendix C). Its items require the transfer of the complexity-based structure learned in the context of gas particles to the collective behavior of ants inside rooms in an anthill. They test far transfer because none of the visual scaffolding for that structure is present in the questions themselves. The coding scheme was developed using a process similar to that described for the near-transfer questionnaire. The items of the far-transfer questionnaire were coded by two researchers according to the detailed coding scheme (Appendix C, Table C.1). For each item, students could get a maximum of four points, thus the maximum total score of this questionnaire is 16. The total score for each student was computed and converted to percent out of 100. A Mann–Whitney test was conducted to compare student scores from the experimental group versus the comparison group. Effect size was manually computed using the squared Z-value of the Mann–Whitney test, divided by (N−1). Open items in far-transfer questionnaires were coded independently by two researchers. Cohen’s (1968) kappa was found to be 0.814. Any remaining disagreements were resolved through discussion.

Comparison between near and far transfer: The two problems, namely the “aquarium problem” and the “ants problem,” were coded by two researchers. The coding was based on the frequency of use of components within the complexity-based structure. Table 2 describes our coding table, based on the complexity-based structure, and examples of students’ responses to the two transfer problems.

The near-transfer and far-transfer problems were independently coded by two researchers, who then compared their results. Agreement was found to be 0.859 on average. These results indicate high agreement between the two researchers. Any remaining disagreements were resolved through discussion.

• 4. Path analysis is unlike traditional regression methods, which assume that only direct associations exist between dependent and independent variables. Path analysis takes into account the indirect factors that play an important role in capturing multiple relationships. In addition, it speculates a unit variance by using standardized path coefficients to enable comparisons of the magnitudes of each variable (Ahn, 2002). In this study, we conducted the path analysis using multiple linear regressions to test for effects of the independent variables, conceptual knowledge, systems understanding, and CT, on the dependent variable, far transfer of learning. Near transfer was not included in this analysis because the item used to test for it is also part of the CT questionnaire. A z-score was computed for all variables.

Findings

In this section, we first present a quantitative analysis of the questionnaire scores for conceptual learning, systems understanding, CT and transfer. We next describe a qualitative analysis of the students’ responses to the near- and far-transfer questions in relation to the element of the complexity-based structure framework. Finally, we present a path analysis to examine the relationships between the three components of conceptual learning, systems understanding, and CT, and examine their impact on the transfer of learning.

Conceptual knowledge, systems understanding, CT and transfer

The first research question explores the impact of constructing models of complex systems on the topic of gases with the MMM platform on students’ conceptual learning, systems understanding, CT, and near and far knowledge transfer, as compared with normative instruction of the subject. We performed a quantitative analysis of students’ conceptual knowledge and systems understanding from the gases questionnaire scores, and of CT from the CT questionnaire scores (Table 3).

Table 3 Conceptual understanding before and after experiencing either the MMM learning unit or the normative learning unit (experimental n = 26, comparison n = 24)

Conceptual knowledge

The results show that both groups displayed learning, but that a higher score was obtained by the experimental group, differing by 17% of the total score. A repeated-measures analysis of variance (ANOVA) shows a significant time effect (F(1,48) = 74, p < 0.01) from pre- to posttest. The interaction between time and group (F(1,48) = 0.13, p < 0.05) indicates the superior learning of the experimental group.

Systems understanding and computational thinking 

The results show that both groups significantly enhanced their understanding of the different systems components (Micro, Macro, Micro/Macro). A repeated-measures ANOVA shows a significant time effect [Micro: F(1,48) = 15.92, p < 0.01; Macro: F(1,48) = 19.56, p < 0.01; Micro/Macro: F(1,48) = 45.85, p < 0.01] from pre- to posttest. The specific component that contributes to this result is the micro-level reasoning regarding the systems. The interaction between time and group at the micro level (F(1,48) = 6.47, p < 0.05) shows the superior learning of the experimental group. It is interesting that although a greater advantage of learning gains was seen for the experimental group in both the macro level and bridging micro and macro levels, the difference between the groups in these cases was not significant.

CT: The results show that both groups increased their CT score from pretest to posttest. However, the experimental group showed a much greater increase than the comparison group (6% vs. 32%). A repeated-measures ANOVA shows a significant time effect [F (1,48) = 50.70, p < 0.01] for both groups. The interaction between time and group is significant (F (1, 48) = 23.10, p < 0.01), favoring the experimental group.

Near and far transfer

Quantitative analysis of students’ posttest scores in the near and far transfer questionnaires shows that for both near and far transfer, the experimental group’s scores were higher than the comparison group. To assess near transfer, we analyzed the posttest scores on the “aquarium problem” for both the experimental and comparison groups. A Mann–Whitney test showed significantly greater scores for the experimental group (Mdn = 62) compared with the comparison group (Mdn = 11), U = 95.5, p = 0.00, and a small effect size of 0.23. To assess far transfer, we conducted a similar comparison of the posttest scores for the “ants problem.” A Mann–Whitney test showed significantly greater scores for the experimental group (Mdn = 50) compared with the comparison group (Mdn = 31), U = 161, p = 0.003, and a medium effect size of 0.36. When comparing near and far transfer, higher scores and effect size were found for the far transfer.

In summary, the quantitative analysis of the Gases questionnaire shows that students in the experimental group significantly enhanced their conceptual knowledge of gases as well as their systems understanding compared with the comparison group. Systems reasoning at the micro level significantly contributed to this result. We further found a significant increase in CT scores amongst students from the experimental group compared with the comparison group. Finally, higher posttest scores for the experimental group were found on both the near-transfer problem and the far-transfer problem.

The contributions of constructing models of complex systems with MMM to students’ near transfer and far transfer

To answer the second research question, regarding which aspects of the complexity-based structure are identifiable in students’ knowledge transfer, we performed a qualitative analysis of the students’ descriptions and explanations for each of the questionnaire items. For the qualitative comparison, students’ written answers were coded according to the complexity-based structure. Focus was placed on features/terms relating to individual entities in the system, in this case fish and ants. The complexity-based structure includes three categories: entities’ properties, their actions, and their interactions with other entities and with the walls. The students’ answers did not include detailed descriptions of the fish and ants’ actions. Therefore, the focus of analysis was on properties and interactions. For each of these categories, the students’ answers referred to changes in speed and in heading of the entities. In some cases, students only mentioned that interaction occurs, without referring to its consequences in terms of heading or speed variables. Thus, for the interactions category of the complexity-based structure, we noted where speed and heading were mentioned or incorporated into a rule. Table 4 illustrates the number of students who used each variable within their responses to the two transfer problems.

Table 4 Students’ responses to the near- and far-transfer problems based on the complexity-based structure, comparing the experimental and comparison groups

Experimental group

Results show that in the near transfer problem, students’ explanations were based primarily on two categories of the complexity-based structure—Properties and Interaction. In the Properties category, 58% of the students addressed the heading of the fish when they moved inside the aquarium. Most students referred to interaction with another fish in their response, describing changes in both speed (85% of the students) and heading (73% of the students) as a result of interactions with another fish. With respect to the interaction of the fish with the edge of the aquarium, 65% of students referred only to the speed of the entity (and not to its heading) when describing the interaction.

For the far-transfer items, most students’ responses focused on the Properties category: 50% of students referred to the ants’ speed and 92% of students referred to the heading variable. Regarding interaction with other ants, 50% of students only mentioned that an ant meets another ant, without referring to this meeting’s impact on its heading and/or speed. Similar results are seen for interactions with the wall. Most students did not refer to the change in speed and heading as a result of changing the size of the room, which in turn affects both interactions between the ants and with the room’s walls.

Comparison group

Results show that students’ responses relied mainly on describing the Properties category of the complexity-based structure categories for solving the near- and far-transfer problems. In the near-transfer problem, 51% of students addressed the heading property of the fish. In the far-transfer problem, in their responses, 54% of students focused on the speed property of the ant, and 71% on the heading property of the ants.

Table 5 provides examples of responses of two students for near and far transfer problems. Two representative students were selected from two groups. ST1 is a student from the experimental group, and ST2 is a student from the comparison group. These students were selected as their responses reflected the responses of their group. Table 5 presents a qualitative analysis of these students’ responses for the near and far transfer items.

Table 5 Analysis of two students’ responses, ST1 (Experimental group) and ST2 (Comparison group), in the near and far transfer problem based on the complexity-based structure components

For both problems, ST1’s response included the entities’ properties mainly describing the heading in both near and far transfer. For interactions, in the near transfer he used both categories of interactions, with another entity (describing the heading and speed), and with the wall (describing only the heading). However, in the far transfer problem, he described only interactions with other entities, relating only to their speed and not their heading. Like ST1, ST2’s responses in both the near and far transfer problems addressed the entities’ properties, describing mainly the heading. However, her use of the interactions categories was less frequent compared with ST1, and limited only to the near transfer problem. In that problem, she mentioned the interaction with other entities and with the walls, but without any indications of the speed and heading components.

In conclusion, a comparison of students’ replies to the two transfer problems showed that in the near-transfer problem, students in the experimental group gave explanations that were mainly based on describing the heading of the fish, the interactions between the fish, and interactions with the aquarium wall. However, in the far-transfer ants problem, students explained their answers based on both the heading and the speed, and only half of the students mentioned the interaction with other ants. The students did not refer to the effect of changes at the macro level on the interaction between the entities and the wall, which in turn may be reflected in changes in the entities’ heading and speed. This result indicates better performance among students in the near transfer problem, where they engage in problem-solving cued and supported by the MMM pseudocode, which includes an explicit representation of the complexity-based structure. However, although the pseudocode was presented to them as well, most students in the comparison group did not rely on it in their answers to the problem. For the far-transfer items, similar to the experimental group, students in the comparison group referred only to the ants’ properties of heading and speed to answer the problem.

The effect of students’ conceptual knowledge, systems understanding, and CT on learning transfer

In this study we used path analysis to study the effects of students’ conceptual learning, systems understanding, and CT on their performance in the far-transfer questionnaire. Because the near transfer items are also part of the CT questionnaire, it is not included as a separate variable. The analysis focuses on the experimental group’s far transfer of learning.

We ran statistical analysis to test for causality between variables. The independent variables are (1) conceptual knowledge, represented by the posttest scores of the Gases questionnaire; (2) systems understanding, represented by the micro-level posttest scores of the questionnaire relating to systems components: this was chosen because learning associated with this item was shown to contribute the most to students’ conceptual learning compared to the comparison group (see Table 2); and (3) CT competence, represented by the posttest scores of the CT questionnaire. The dependent variable is the far-transfer score, as rated in the far-transfer questionnaire.

Figure 4 illustrates the results of the analysis, using multiple linear regression to test for effects of posttest Gases scores, posttest micro-level scores, and posttest CT scores on the independent variable—the far-transfer scores. The results show the direct effect of the two variables—posttest micro-level and posttest CT—on the transfer score [${R}^{2}=0.4, F\left(\mathrm{2,23}\right)=4.86, p=0.01$]. Significant results were found for both the posttest micro-level score (β = 0.64, p = 0.012) and for the posttest CT score (β = 0.41, p = 0.03). An indirect effect of the posttest Gases score on the transfer score was also found [${R}^{2}=0.5,F\left(\mathrm{2,23}\right)=11.17, p=0.00$]. This indirect effect occurred throughout, affecting the posttest micro-level score (β = 0.73, p = 0.000).

Fig. 4

Causal paths with statistically significant direct effects. CT Computational thinking

Results show that only some of the hypotheses were confirmed: (1) involving students in learning using the “Modeling Gas Behavior with MMM” learning unit promotes conceptual learning, which impacts systems thinking mainly at the micro level; (2) learning about systems at the micro level has a positive effect on students’ performance in the far-transfer tasks; and (3) increased CT has a positive direct effect on students’ performance on the far-transfer tasks. One hypothesis was disconfirmed: increased CT does not impact conceptual knowledge or systems understanding. Thus, conceptual learning using the “Modeling gas behavior with MMM” learning unit indirectly impacts the transfer of learning through systems thinking but is independent of CT, which, in turn, contributes to learning transfer.

Discussion

This study explored the possible synergy between learning science, computational thinking, and a complex systems perspective. In that context, it also addresses a construct—transfer of learning—that has only rarely been researched in the past decade, especially in schools and among students who are learning science. The design of the focal learning environment sought to encourage learning transfer by guiding students’ modeling toward a complexity-based structure, which can be generalized across many phenomena. The research examined students’ conceptual learning, systems learning, learning of CT, and near and far transfer of learning. In all categories, the experimental group outperformed the comparison group. Therefore, one main conclusion of this study is that we found a higher degree of learning transfer among the experimental group students than among students from the comparison group. The second conclusion is that the main contribution of a complexity-based structure is to students’ understanding of the properties and interactions of entities at the micro level of the system. The third conclusion pertains to the independent contributions of developing a complex view of scientific phenomena and learning CT to the far transfer of learning. These conclusions are discussed in the following sections.

The contribution of model construction to promoting CT, systems understanding, conceptual learning of science, and knowledge transfer

In answer to the first research question, our findings show that, compared to students who experienced a normative curriculum, students who constructed computational models significantly improved their science conceptual knowledge, CT, their systems understanding. This result is similar to the results of much of the research on learning about complex systems by constructing agent-based computational models (Basu et al., 2014; Wagh & Wilensky, 2018). More specifically, regarding systems understanding, the main contribution to learning involves the micro level of the system, a result that replicates much previous research. We saw this result in students’ scores on the conceptual knowledge questionnaire, which presents challenges across the range of science concepts and systems levels. We also saw it in the qualitative analysis of students’ explanations on the transfer items, where the strong differences involved noticing interactions among entities in the system.

The micro level is addressed in most normative teaching; however, the way it is presented—as pictures or animations—is not as powerful as constructing and exploring agent-based models (e.g. see the textbook Material Science (Ben Horin, Orad & Welger, 2013). Agent-based models are programmed at the micro-level of the system’s entities, making the micro level the main form of access to the modeled system (Bar-Yam, 2003). Moreover, the learning units in this and other studies not only foreground these entities in the code, but also draw students’ attention to the single entities’ behaviors (Samon & Levy, 2017; Dickes, Sengupta, Farris, & Basu, 2016; Sengupta & Wilensky, 2009). It makes sense that having students define the agents’ behaviors by coding would result in their explanations incorporating such behaviors. To summarize, the finding that, compared with normal teaching, the students in the experimental group more prominently improved their understanding of the micro-level of the system, reflects both the differences in focus in the two groups’ curricula and the advantages of the MMM platform’s incorporation of a complex systems approach.

In this study, we go one step further and claim that engaging in the construction of models with the MMM platform not only contributes to promoting students’ conceptual learning and systems understanding, but also facilitates their CT. This result addresses the importance of providing students with the visual scaffolding, namely the complexity-based structure, that guides students’ construction of the models and thus contributes to facilitating their CT too.

With regards to transfer, our findings showed higher rates of transfer in the experimental group than the comparison group, in both near and far learning transfer. Similar to Fuchs et al. (2003), our results show that the effect size for far transfer was higher than that for near transfer. The higher degree of near transfer in the experimental group aligns with other theoretical and empirical studies that have argued for the achievability of near transfer (Barnett & Ceci, 2002; Day & Goldstone, 2012; Denning, 2017; Falloon, 2020; Gentner, 1983; Hummel & Holyoak, 2003; Klahr & Chen, 2011). Providing students an explicit scaffold for organizing their problem-solving efforts during the original learning phase helps them relate this same scaffold to new situations, thus increasing the rate of learning transfer (Catrambone & Holyoak, 1989; Fuchs et al., 2003; Rosholm et al., 2017; Terwel et al., 2009).

The complexity-based structure was used in our study to tailor the students’ efforts in modeling by compartmentalizing the code blocks into three groups, which are commonly used in ABM of complex systems: properties, actions, and interactions. The students from the experimental group used this structure to construct several models of gas-related phenomena during the learning unit, so that it may have gradually proved its utility to them. The near transfer item used the same complexity-based structure to describe pseudocode for modeling a phenomenon that was not learned in the unit. Thus, the condition of explicit scaffolding is fulfilled.

The contribution of the complexity-based structure to learning transfer

In answer to our second research question, we found that the differences between the experimental and comparison groups regarding this item are not only in quantity, but also in quality. Our qualitative analysis of students’ answers to the near transfer item revealed a distinct explanation pattern. The experimental group students described the scenario presented in the question and their predictions using the system’s entities, their properties, and the interactions they undergo, similar to the complexity-based structure. In contrast, the comparison group did not attend to the interactions and properties, which were in the pseudocode—ignoring them and basing their explanations primarily on the entities’ actions.

Several studies have claimed the difficulty of achieving far transfer (Barnett & Ceci, 2002; Denning, 2017) and others have argued that only a very limited degree of far transfer can occur (Chi & VanLehn, 2012; Sala & Gobet, 2017). These papers explained that the difficulty of achieving far transfer is related to the high impact of surface features on reasoning when expertise is lacking. Learners do not note similar deep structures, which are not apparent, and they are swayed by the dissimilar surface features (Chi & VanLehn, 2012; Gick & Holyoak, 1983). However, the experimental group in our study showed a medium effect size for far transfer.

Two theoretical approaches implemented in this study may have been useful in promoting learning transfer. One is a complex systems approach, which emphasized the importance of looking at the many components that make up a system and finding the rules that describe their behaviors and interactions as an explanation of emergent patterns (Bar-Yam, 2003). The way this is implemented in the MMM platform is by (1) using separate sets of code for each population (e.g., diffusion of two kinds of molecules inside a container, atoms, and electrons inside a conductive wire); (2) organizing the coding actions into complexity-based compartments—properties, actions, and interactions—for each population; and (3) designing coding blocks that highlight similarity across phenomena; for example, what happens when two entities meet? There is one block for such interactions, with a menu that enables students to choose among several actions that take place for different phenomena, such as collide to describe particles in a gas, or attract to describe the formation of a liquid from gas.

The second theoretical approach is related to CT, a form of reasoning that generalizes explicitly by using content-free computer language to describe and construct explanations (Wing, 2006). This is applied in MMM by having students construct the models from code segments, which are represented as blocks that can be dragged into the program. These blocks have different computational relationships among them, such as condition-action rules (e.g., what should a gas particle do when it hits a wall) and iterations (e.g., at every time-step, a particle needs to check whether it’s about to collide). As described above, the behavior of many systems in chemistry and physics is collapsed in the platform into a very small set of blocks, where the distinctions between phenomena reside in two parts of the program and model. One is the macro-level layout of objects, such as pipes and containers. The second is the action segment of the program’s conditionals, whether you ignore, repel, attract, or collide with another particle when you collide with it.

It would seem that this combination of two general representations with using the MMM platform to create computer models of science phenomena helped some of the students see beyond the topics of the learning unit. As we have seen in the qualitative data regarding the far-transfer items, unlike the comparison group, the experimental group students indeed noticed and mentioned two categories from the complexity-based structure—the properties of entities in the system and the interactions between them. This shows us that what carried over from the source to the target phenomena is this general form of thinking. Regarding CT, we do not have similar data to examine whether this carried over into the target phenomena; however, in the next section we describe its overall impact on transfer.

A comparison between our approach to encouraging transfer and the methods described in the research literature revealed only a very partial overlap. One method emphasized in the literature, for example, is direct instruction, which explicitly identifies and emphasizes the underlying deep principles that the original context and the target context have in common (Fuchs et al., 2003; Rosholm et al., 2017; Salomon & Perkins, 1989; Terwel et al., 2009). In our study, the students used the complexity-based structure to construct models, but there was no explicit teaching about how this same structure could be used to model other systems. Nevertheless, we found that this indirect instruction led to the medium effect size of far transfer. A second method of increasing learning transfer is to teach the source domain in depth (Chi & VanLehn, 2012; Lobato, 2006; Marton, 2006). The learning activities in the present research would seem to conform to this stipulation of depth. Learning by modeling phenomena involves deep processing of the learned concepts, their instantiation in a variety of settings, and their creative integration into a working experiment or device. Thus, while direct instruction was not used, learning in depth is typical of the instructional unit, as also seen in the greater conceptual learning among the experimental group. A third method of increasing transfer of learning is scaffolding learners when they are engaged with the target problems, so that they can see the similarity with their previous learning of the source problems (Catrambone & Holyoak, 1989). This was not done in the present study; the target problems were part of questionnaires at the end, and the students were not supported in this respect.

To summarize, of the known methods to increase transfer, we applied only teaching the source domain in depth. However, it would seem that a central theoretical contribution of this work is to offer a fourth method, using visual epistemic scaffolds of the general thinking processes we wish to support and incorporating these visual structures into the core problem-solving activities. Based on Malkiewich and Chase (2019), we argue that providing students with the visual epistemic scaffold, namely the complexity-based structure, enables them not only to notice but also to focus on the deep structure of the phenomena they are modeling. This affordance can be another explanation of our result related to medium effect size of far transfer.

The contribution of CT, conceptual learning and systems understanding to learning transfer

To answer the third research question, we used path analysis to explore possible dependencies between the different forms of learning, CT, systems understanding, conceptual learning, and knowledge transfer. Similarly, Zhang et al., (2020) addressed learning by constructing computational models, and used path analysis to test the relation between CT, engineering learning, and conceptual science understanding. However, our study differs from Zhang et al., (2020) by bringing the learning transfer construct to the fore. We explored the contributions of CT, understanding of complex systems, and conceptual learning to learning transfer within a learning environment that is based on the construction of computational models. This analysis revealed that science conceptual learning, understanding of complex systems, and CT all have a positive impact on learning transfer, when the learning is based on model construction. However, we also gained two more important insights that contribute to understanding the interactions between these multiple forms of learning.

One is that, in the context of our study, learning CT and learning how to think complexly made independent contributions to learning transfer. Even though the learning design includes both creating computational objects and arranging them according to complexity-based categories, the effects of these two design decisions are independent of each other, so that they have a cumulative effect on the far transfer of learning.

In conclusion, the design of the visual representation of the complexity-based structure in the MMM interface contributes to the significant independent contribution of both CT and systems understanding to learning transfer. Based on this, one may surmise that using either one of the design decisions—enhancing CT or enhancing thinking through the lens of complex systems—could increase learning transfer, a conjecture that could guide further research.

A second insight is that conceptual understanding in science impacts learning transfer only through a particular perspective—whether the students understand the micro-level behaviors of the particles, or individual entities in the system. Thus, while previous research has shown that deep teaching of the source topic can enhance learning transfer (Chi & VanLehn, 2012; Lobato, 2006; Marton, 2006), in our study, an additional stipulation is added, which relates to the structure of knowledge about systems: when teaching about systems in science, framing them with a complexity-based structure is a powerful way of increasing learning transfer. It remains to be tested whether these two insights can be generalized in additional settings.

Conclusions, limitations and future work

The study focused on interweaving CT and science through the construction of computational models with MMM, and sought to test the transferability of the tool’s complexity-based structure across contexts. Our findings reveal the significant effect of this synergy on the enhancement of conceptual learning, systems understanding, and CT. In addition, combining these three factors both directly and indirectly contributes to learning transfer of the general, complexity-based structure. At the theoretical level, this study contributes by providing a method for increasing learning transfer by using visual epistemic scaffolds of the general thinking processes and integrating these visual structures into the core problem-solving activities. This is particularly true when bringing to the front the synergy between CT and complex systems thinking to learn about diverse systems in science.

One limitation of this study is the restricted duration of learning, which in turn revealed only a medium effect size of far transfer. Because of COVID-19, students from the experimental group engaged in only four of six 1.5-h sessions, less than we had planned. This limitation prevented us from comparing earlier and later construction activities to explore the development of CT through modeling with MMM. We believe that facilitating CT and transfer of learning may require more time and more opportunities to engage in the construction of models with MMM.

Another limitation, which is also related to the restricted duration of learning, is the limited sample of students in both groups, again due to COVID-19. We think that analyzing a larger sample of students’ responses to near and far transfer problems may provide us with more insights related to the core components of the complexity-based structure students transfer and use in the transfer problems.

A third limitation is that the same regular teacher did not teach both groups on her own. It is important to mention that the regular teacher had no experience in teaching with MMM, and due to the pandemic, she did not have enough time to learn the tools and pedagogy before conducting the study. In addition, rather than using an experimental design, such as splitting the classes in half, the study used a quasi-experimental design in which the classes were left intact. This design could have introduced a bias, for example, if one teacher was more effective in teaching than the other teacher.

The fact that the data analysis focused on students’ post-test scores and responses in the transfer questionnaires is another limitation, assessing only the students’ learning outcomes. Future work may include analyzing students’ constructed models and their learning process (through observations, and responses to the worksheets) to explore how knowledge is developed and transferred across contexts.

This study is based on comparison between students who learned by constructing computational models using a computer-based tool and students who learned with the normative curriculum, which did not include any use of computer-based tools. Other studies may explore how learning transfer would be promoted by comparing between two approaches to learning by constructing models versus exploring pre-built models.

Finally, learning transfer was not directly instructed in the “Modeling gas behavior with MMM” learning unit and in the normative science curriculum. In future research, it would be interesting to compare near and far transfer results with the use of direct instruction (Fuchs et al., 2003; Salomon & Perkins, 1989; VanLehn & Chi, 2012) to help students abstract the complexity-based structure from its concrete representation in the MMM and use it in other domains. Direct instruction could also explicitly illustrate the practice of modeling based on the complexity-based structure. It would provide students with several phenomena from diverse domains, all of which can be modeled by relying on the complexity-based structure.

In our own future work, we will continue to refine the learning design of this unit, and to expand the implementation of the platform to the instruction of other topics, in order to test whether the same conclusions are reached and to deepen our understanding of the learning process. Other research could compare our results with those of studies that employ computational modeling platforms with a different visual scaffold, to further explore the significance of employing the complexity-based structure.