Understanding of Relation Structures of Graphical Models by Lower Secondary Students

Abstract

A learning path has been developed on system dynamical graphical modelling, integrated into the Dutch lower secondary physics curriculum. As part of the developmental research for this learning path, students’ understanding of the relation structures shown in the diagrams of graphical system dynamics based models has been investigated. One of our main findings is that only some students understand these structures correctly. Reality-based interpretation of the diagrams can conceal an incorrect understanding of diagram structures. As a result, students seemingly have no problems interpreting the diagrams until they are asked to construct a graphical model. Misconceptions have been identified that are the consequence of the fact that the equations are not clearly communicated by the diagrams or because the icons used in the diagrams mislead novice modellers. Suggestions are made for improvements.

Introduction

There are several reasons for introducing computer modelling in secondary education. It fits the goal of letting students develop scientific literacy. As citizens, students must learn about the possibilities and limitations of computer models, as future professionals, they should learn how to use and construct models. In addition, computer modelling enhances science education in the sense that it enables students to study subjects that are more realistic than the scholarly problems found in textbooks. With computer models, subjects can be studied that have a complexity surpassing the mathematical capabilities of the students. The importance of computer modelling for education was already recognised in the eighties of the twentieth century (see, for instance, Ogborn and Wong 1984). Computer modelling became formally part of the Dutch upper secondary physics curriculum in 1991. Though, a concrete implementation of a complete, well-designed and thoroughly tested learning path on quantitative computational modelling into the physics curriculum did not yet exist in 2008 (Lijnse 2008). Therefore, we started a design research project in which such a learning path was developed and tested in school practice.

As Schecker (1998) points out, computer modelling requires a new way of thinking which takes considerable time to learn. Also, for computer modelling, many abilities are required, such as the ability to interpret a realistic situation in terms of physical quantities, competencies with respect to formulas and variables, graphs, and the use of a computer modelling environment, and competencies with respect to the evaluation of models. Hmelo-Silver and Azevedo (2006) note that the complexity of modelling includes the amount of domain content to be modelled, the granularity of the model depiction required by the modelling task, the level of abstractness of the model representation, the affordances available to the students, and difficulties of students in perceiving and understanding model output. The cognitive load for development of the required abilities is by its nature high for students. In order to diminish this load, it may be advantageous to start with a modelling learning path early in a student’s school career, in which all these competencies can be developed in a well-balanced way, instead of treating computer modelling as a stand-alone instructional unit at upper secondary level. An early start gives students more time to get used to modelling. Getting used to this new way of thinking might also be easier for young children than accommodating to models and modelling at a later stage. Another advantage of starting early is that computer modelling offers new ways of teaching and learning physics that may be more effective than traditional instruction. It may be more efficient if the modelling learning path and the standard physics curriculum are well tuned to each other. Therefore, we decided to start the learning path from the initial phase of physics education (age, 13–14 years) and to completely integrate modelling into the curriculum, with a focus on computer modelling.

One of many decisions that had to be made for the design of the modelling learning path is the choice of a modelling approach. We chose the graphical system dynamics approach developed by Forrester (1961), because of its wide range of applications to phenomena that can be modelled as systems whose states change over time. Henceforth, we refer to this approach as ‘graphical modelling’. Several computer environments exist in which this approach has been implemented. Examples are STELLA (Steed 1992), Model-It (Jackson et al. 1996), Co-Lab (Van Joolingen et al. 2005) and Coach (Heck et al. 2009b). These systems offer alternative ways of specifying a model with varying degree of black-box to glass-box exploration and modelling, and with various levels of instructional support and scaffolding, but they all adhere to the same basic principle: the students specify models drawn as graphical structures that can be executed (simulated). We selected the Coach environment because it is one of the few integrated computer learning and multimedia authoring environments in which computer modelling can be combined with measurements through video and/or sensors, and with computer animation. The use of only one computer learning environment for multiple purposes reduces cognitive load for novice learners. The combination of experimentation and modelling is expected to help students in making connections between the concrete realistic situations that are modelled and the more abstract models and model output (Sander et al. 2002). The implementations of graphical modelling in Coach and in STELLA are similar in most aspects. A practical advantage is that Coach is available at the majority of Dutch secondary schools.

Our choice for graphical modelling is also rooted in theories of cognition and instructional design, and in research on systems thinking. Characteristics of system thinking include perceiving a system as something consisting of many elements that interact with each other, understanding that a change of one element in a system may result in changes in other elements or even the whole system, and embracing that the relatively simple behaviour of individual elements may be aggregated through some mechanism (e.g. cause and effects) to explain the system at the collective level (Wilensky and Resnick 1999). System thinkers claim that this approach promotes and supports the causal reasoning skills of students. From a mathematical perspective, the elements of a system are variables that are related either by semi quantitative relationships (e.g. as variable t increases, variable x decreases) or by quantitative relationships, i.e. mathematical equations (e.g. difference equations and direct relations). In a graphical model, the variables and relationships between variables are visually represented as a system of icons in a diagram. Several researchers (e.g. Niedderer et al. 1991) have suggested that the visual representations in graphical models provide students with an opportunity to express their own conceptual understanding of physical phenomena and can help to shift the focus from learning and working with mathematical formulas to more qualitative conceptual reasoning. The graphical model can serve as a representation of an internal (mental) model of a modeller and it provides a means for communication and analysis. According to Fuchs (2008), graphical modelling reflects fundamental aspects of figurative human thought and therefore supports analogical reasoning. He considers graphical modelling as easy and powerful enough to lend itself to explicit modelling in classroom. From the perspective of cognitive load theory (Sweller 1994; Sweller et al. 1998, 2011; Van Merrienboer and Sweller 2005), which relates problem and presentation formats to learning processes, cognitive load, and transfer, success of graphical modelling can be attributed to the assumption that the use of a graphical external representation supports the off-loading of working-memory and allows the freed working memory to be used for learning.

Research has shown that students using an environment for graphical modelling can reason qualitatively and intuitively about systems (Doerr 1996), and graphical modelling seems to be effective for learning to reason with complex structures (Van Borkulo 2009). Schecker (1998) reports that half of his students after a mechanics course with STELLA were able to construct a qualitative causal reasoning chain on a new subject. Mandinach and Cline (1996) note positive effects of modelling on students’ cognitive and motivational processes. Kurtz dos Santos et al. (1997) report transfer from one modelled domain to another. All this gives ample support for the advice of Savelsbergh (2008) to consider graphical modelling as an appropriate candidate for a modelling approach in the renewed Dutch secondary science curricula.

However, Löhner (2005) reports that, in spite of the claims of proponents of modelling approaches, evidence to support the claims of learning by modelling is still scarce, especially when it comes to experimental studies. Graphical modelling is still not without problems in education practice. Several authors report on difficulties that students have, especially when designing or adapting graphical models (Tinker 1993; Bliss 1994; Kurtz dos Santos and Ogborn 1994; Sins et al. 2005; Lane 2008; Westra 2008; Van Borkulo 2009; Ormel 2010). Groesser (2012b) points out that the information provided by system dynamics models can only benefit students who are familiar with system dynamics methodology and who are thus able to read and interpret graphical models. It is not clear yet if the hiding of mathematical details in the graphical approach, often seen as an advantage in the modelling process, actually is effective. Many research studies report on obstacles that students encounter in graphical modelling, but few focus on the question of how diagrams and icons are actually understood (Doerr 1996; Lane 2008) and which icons best convey the nature of the variables (Lane 2008). In this paper, we intend to fill this gap.

In the design of the modelling learning path for lower secondary physics students, we have addressed as many known difficulties and obstacles that students may encounter in their first steps into graphical modelling. The instructional design has been guided by principles of cognitive load theory. According to this theory, the cognitive load of a learning task consists of three parts: (1) intrinsic load of the subject matter, which cannot be altered by instructional methods, (2) extraneous load that reflects the effort to process the instruction, and (3) germane load that reflects the effort for the acquisition of schemas that can be stored in long-term memory and reduce working memory load (Sweller et al. 1998). Goal of our instructional design is to reduce extraneous cognitive load and optimize intrinsic and germane load. Because of the high complexity of modelling, the segmenting principle is a fundamental part of our instructional approach (Mayer and Moreno 2010). We applied this principle by subdividing the learning path in five intertwined partial learning paths, on (1) the computer environment, (2) graphs, (3) formulas and variables, (4) graphical models, and (5) evaluation and general nature of models. In this paper, we focus on only one particular aspect of only the fourth partial path, namely, on how students understand the relation structures shown in the diagrams of graphical system dynamics based models. We consider a correct understanding of these structures as a necessary (but not sufficient) condition for understanding graphical models and modelling (cf. Morgan 2001); the step from understanding of the structure towards an understanding of entire models goes beyond the scope of this paper. In subsequent sections, we discuss approaches to graphical modelling, our research methodology, the design and implementation of our learning path on graphical modelling, and research outcomes.

Graphical Models and Graphical Modelling

General Description

In Fig. 1, an example of a graphical model is shown; it is about the motion of an object falling through air. The graphical equivalent of a difference equation is the combination of a ‘stock variable’, represented by a rectangle, and one or more ‘flow variables’, represented as ‘thick’ arrows. The terms ‘stock’ and ‘flow’ are used because the diagrams often can be understood metaphorically as flows into or out of a stock (‘inflows’ and ‘outflows’). Each stock needs an initial value, as a starting point for numerical integration. Variables that are not explicitly part of a difference equation are referred to as ‘auxiliary variables’ and are represented by circular icons. Apart from difference equations, direct relations appear in the models. By a direct relation, we mean a mathematical relationship between symbolized quantities in which at least one quantity can be isolated and written as a closed form expression of the other quantities. The direct relations must be entered explicitly into the computer environment, otherwise a run of the computer model is not possible. ‘Connectors’ (thin arrows) indicate the presence of the direct relations, but they are not immediately visible: the modeller must click on an icon or put the mouse over an icon to see a formula.

Fig. 1

Graphical model for the velocity v of an object falling through air. As the velocity increases due to the force of gravity F _grav, the air friction F _air increases too. As a result, the acceleration a diminishes until the net force F _net becomes zero and a constant velocity is reached. The graphical model corresponds to the difference equation Δv = a·Δt, and the direct relations a = F _net /m, F _net = F _grav − F _air, F _grav = m·9.81 and F _air  = k·v ². In this model, v is a stock variable, a is a flow variable and all other quantities are depicted as auxiliary variables

Approaches to Graphical Modelling

Goals of the Diagrams

Forrester (1961, p. 81) considers the diagrams as “an intermediate transition between a verbal description and a set of equations.” Their main goal is to communicate the causal assumptions and the main features of the model. The diagrams are used in two different ways: namely as

• 1. A graphical expression of a mental model. This graphical expression (the diagram) can subsequently be turned into a quantitative computer model by entering the required direct relations.

• 2. A way of expressing the meaning of the equations and of the structures of equations. Here, equations can form the starting point for model construction and the diagram clarifies the meaning of the equations.

Stock-and-Flows Approaches to Graphical Modelling

In more traditional approaches to graphical modelling, stocks, flows, and auxiliary variables are considered as the fundamental building blocks. Herein, students must first determine the type of a variable. Often, the stock-flow metaphor is used as an aid in designing and understanding of graphical models. This is expected to be easier than establishing and understanding the equations, but in educational practice such an approach does not go without problems. The following problems have been identified:

• The distinctions between flows and stocks are not easily made by students (Tinker 1993; Bliss 1994; Schecker 2005; Westra 2008).

• Considering the type of a variable as a fundamental property is questionable: one model variable can be part of several equations, in each of which its type can be different.

• According to Bliss (1994), the representation of the rate of change of a variable as itself another variable is problematic for students of age 12–14 years. When not confident of this idea, students cannot express themselves with stock-flow diagrams.

• Students of age 16–18 years tend to use objects, events, and processes instead of true variables (in the sense of measurable properties) when trying to conceptualize their models, although this seems to depend on the problem that is modelled (Kurtz dos Santos and Ogborn 1994).

• Not all students have sufficient prior knowledge of the flow of a fluid through a valve and its accumulation in a tank to understand this as a metaphor for stock-flow diagrams (Tinker 1993).

• After an instruction based on this metaphor, the stock-flow diagrams are not yet easily grasped by many a student (Tinker 1993; Sins et al. 2005; Lane 2008).

• The stock-flow metaphor is not very useful for modelling dynamics, as it is hard to imagine ‘position’ as a stock where ‘velocity’ accumulates (Tinker 1993).

• Even if the use of differential equations can be circumvented, the use of direct relations cannot be avoided. Lower secondary students can have difficulty using these relations if they do not have the required notions of variable and formula (Van Buuren et al. 2012).

• Students may think that variables only depend on variables to which they are directly linked (Van Buuren et al. 2011).

• Constructing models using a qualitative, dependency based approach may lead to less adequate relational structures, as is demonstrated in Fig. 2 (Van Buuren et al. 2011).

  Fig. 2

  

  Example of an incorrect model structure as a result of a dependency based approach. The acceleration a of a body moving through air can be said to depend on the friction force F _air and on its velocity v, leading to this diagram. But the acceleration depends on the velocity via the friction force

A Relation Approach to Graphical Modelling

The question is whether these problems also arise when the equations form the starting point in the creation of graphical models. In this approach, the relations between the variables, i.e. the difference equations and direct relations, are the dominant building blocks of models, instead of the individual variables. This approach possibly fits better to the existing physics curriculum, in which formulas are introduced and used almost from the beginning: it connects to an already existing conceptual network and may enhance the development of mental models. We call this approach a relation approach. A similar approach has been successful in upper secondary mathematics education (Heck et al. 2009a). Central to the relation approach are the link between a difference equation and a stock-flow diagram, and the interpretation of connectors as indicating structures of direct relations.

Interpretation of Diagrams

In order to find out to what extent students are able to interpret the diagrams, first, we reflect on what exactly can be derived from the diagrams. In the diagrams, two different subsystems are used: a subsystem representing direct relations and a subsystem representing difference equations.

• Subsystem 1: relation structures formed by connectors.

  • 1a. Connectors show which model variables are defined by which other variables. This is the key feature of connectors. If a connector points from a first icon to a second, this means that the variable corresponding to the first icon is part of the formula that is used to define the variable corresponding to the second icon. For shortness, we refer to such a first icon as a defining variable and to such a second variable as the defined variable. Connectors do not give information about the exact forms of direct relations.

  • 1b. Especially, if no connector is pointing to a specific flow icon or auxiliary icon, the corresponding model variable is defined by a number.

  • 1c. The structure of icons and connectors provides information about dependencies in a model. If in such a structure it is possible to go from one icon to another by following a chain of connectors and icons, this means that the other icon depends on the first. Van Buuren et al. (2011) introduce the terms ‘directly linked’ for variables that are directly connected to each other by means of one connector, and ‘secondarily linked’ in case there are one or more variables in between (Fig. 3).

  Fig. 3

  

  Variables a and b are directly linked to y, and variables c and d are secondarily linked to y

• Subsystem 2: stock-flow diagrams.

  • 2a. A difference equation can be translated into a stock-flow diagram, and vice versa, under the condition that the modeller knows which variable is the independent variable (the variable of integration, usually time).

  • 2b. Because graphical models are essentially about numerical integration and not differentiation, the relation structure in stock-flow diagrams is always the same: the stock depends on the flows that are connected to it, but a flow does not need to depend on the stock. The value of a stock also depends on previous values of the stock. The flows determine the change of the stock.

Even in a relation approach, students may still have difficulties interpreting the diagrams:

• In physics and mathematics, formulas can be inverted at will, to calculate each of its components, depending on which components are given in advance. But in a graphical model, a formula is used to calculate only one of its components. Which component is calculated is completely determined by the model structure. This is not straightforward; students may think that they are free to choose which component is calculated by the formula (Van Buuren et al. 2011). Thus, students may have difficulty understanding points 1a, 2a and 2b. We refer to this as the ‘relation inversion conception’.

• An error that may follow from this relation inversion conception is the use of circular definitions: within one model, one direct relation is used twice. For example, students may define the acceleration a as F _net /m while simultaneously defining F _net as m·a (Van Buuren et al. 2011).

Main Research Question and Research Methodology

The main research question is

• How do students understand the relation structures used in graphical system dynamics based modelling after a relation-based instruction?

The term understanding is operationalised as being able to derive the information provided by the diagrams as described in “Interpretation of Diagrams” section and to construct or extend graphical models on the basis of given formulas.

We applied an educational design research approach (Van den Akker et al. 2006): instructional materials were designed, tested in classroom and redesigned in several cycles. The research had an explorative character. The reason is that graphical modelling with young students being fully integrated into the entire lower secondary physics curriculum, is new, to the best of our knowledge. We applied several research instruments to monitor the learning processes. Data sources were audio-recorded participatory observations during the lessons and in-class impromptu interviews, student results of modelling tasks (writings, computer results, and screen recordings) and regular tests. We did not feel a need for applying the classical instrument of in-depth interviews from a representative sample because research was carried out in Montessori education, in which learning processes of students are permanently monitored and in-class impromptu interviews are common. The students’ answers to the regular test questions about all aspects of the diagrams are the primary sources for answering the research question. Answers of a student within one test can be expected to be coherent. This qualitative coherence gives insight in to the students’ understanding and is therefore studied. The students’ answers are listed and analysed on the basis of three criteria: (1) which subsystem is addressed? (2) what is the type of the relation between the model variables? and (3) is it an interpretation or construction question?

In line with “Interpretation of Diagrams” section, the subsystem can be either the structure formed by connectors or the stock-flow diagram. Because these subsystems are not necessarily disjoint, in the analysis, we also pay attention to the effects of combining the subsystems. The type of relation between model variables is described in terms of occurrence of variables in definitions and in terms of dependency between variables. For relation structures formed by connectors, this means the distinction between directly and secondarily linked relations; for stock-flow diagrams, this means the translation from a stock-flow diagram into a difference equation and the dependency of (the change of) the stock on the connected flows. The combination of subsystems also makes a clear distinction between difference equation and direct relation important.

The other sources of data are used to increase trustworthiness of the findings. The quantification of the quality of the student answers was used in each developmental cycle to improve the instructional design. It is not applied to measure the progress in students’ performance by statistical methods. The main reason is that, in our setting, the students were stimulated to collaborate and the teachers were still learning how to deal with unexpected student obstacles. In this realistic setting, student results can be expected to be mutually dependent to a certain degree; therefore, the validity of statistical analysis is questionable.

Design of the Partial Learning Path and Overview of the Implementation

Design Considerations

The instructional design has been guided by principles of cognitive load theory to reduce extraneous cognitive load and optimize intrinsic and germane load for novice learners to enable development of deeper understanding (see Plass et al. 2010). Here are a few ways we have attempted to reduce the cognitive load: we subdivided tasks and contents into manageable chunks (segmenting principle). We gradually increased the number of elements in the graphical model that the students were able to manage (sequencing principle). Students received worked examples and partially completed problems to go through for initial cognitive skill acquisition (worked-example and completion principle). We provided students with prior instruction about behaviour of complex system’s components before presenting the whole system (pretraining principle). We let students work with examples from various physics and non-physics subjects (the variable examples principle).

Results from field testing of pilot versions of the learning path were also used for the design. Changes based on early classroom experiments are the following:

• Because of the importance of difference equations as fundamental building blocks, Δ-notation was used almost directly from the start of the learning path. This was actually a recommendation of students confronted with a change in notation at a later stage. They declared that using Δ-notation from the beginning would be easier and less confusing.

• Because lower secondary students’ notions of formula and variable turned out to be insufficient for graphical modelling (Van Buuren et al. 2012), students get operational definitions of formula and variable in an early stage, before the formal introduction of the two subsystems used in the diagrams.

• Because of students’ difficulties with finding a construction order (Van Buuren et al. 2011), we offered them a general construction plan for graphical models based on the relation approach in accordance with the goal-free principle. We advised them to

  • 1. Collect or establish all formulas for the situation to be modelled.

  • 2. Create the stock-flow diagrams corresponding to the difference equations.

  • 3. Proceed with defining the variables in the diagram that have not yet been defined.

For step 3, it was important that students understood that the stock-flow diagram automatically determined the difference equation by which the stock was defined, but that all flows needed to be defined explicitly, by entering a direct relation. Also, students must understand that circular definitions are not allowed.

• In order to diminish cognitive load, both subsystems were not introduced simultaneously, but separately, in successive modules in the learning path. Eventually, students must learn to combine both sub systems. A pilot study with a group of 23 students, indicated that this was difficult for the students to achieve. This followed clearly from their answers to the multiple choice questions described in Fig. 4. After an instruction in which both subsystems were explained, none of these students selected both correct options.

  Fig. 4

  

  Choose the two options that can correspond to the diagrams. Options B and F are correct

Overview of the Design and Implementation of the Partial Learning Path on Graphical Models

In the design, six phases are distinguished, labelled 1 to 6.

• 1. Orientation phase. In this phase, students are provided with several direct relations and difference equations; Δ-notation is used from the beginning. They also get acquainted with basic features of the computer learning environment and orientate themselves with numerical integration. Learning goals in this phase are orientation and learning-to-use, rather than understanding at a structural level.

• 2. Pretraining phase. All elements that are prerequisites for understanding diagrams and their relationship with formulas are introduced one by one at a more formal level.

• 3. First structural introduction phase. In this phase, the focus is on subsystem 1, the structures formed by connectors. It starts with the introduction of the key feature of connectors (see “Interpretation of Diagrams” section).

• 4. Intermediate phase. For an understanding of graphical models, an understanding of the distinction between difference equations and direct relations (or ‘Δ-formulas’ and ‘direct formulas’, as they are called in the instructional materials) is required. Therefore, in this phase, both types of formulas are introduced formally, making use of examples taken from preceding parts in the learning path.

• 5. Second structural introduction phase. The relationship between stock-flow diagrams and difference equations is clarified in a more structural way, and the two subsystems are coupled.

• 6. Final phase. The general construction plan is introduced and used by the students to construct a more extended model in the computer learning environment, based on a given set of equations.

The principle setup of the partial learning path on graphical models and the distribution of the phases over the modules of the physics curriculum is outlined in Table 1.

Table 1 Overview of the distribution of the design of the partial learning path on graphical models over the curriculum

Overview of the Relation Approach in de Modules Sound and Force and Movement

In this paper, the focus is on the structural introduction phases. This concerns the modules in Sound and Force and Movement. The phenomena that are modelled, sound beats and falling with air resistance, are typical examples with a high level of reality that usually are not part of the lower secondary physics curriculum because the mathematics surpasses the students’ traditional mathematical capabilities.

Design of the Module Sound

For the formal introduction of the subsystem based on direct relations and formed by connectors (subsystem 1), to minimise extraneous load, we need a model that consists of direct relations only and contains no stocks and flows. A model for sound beats caused by two tuning forks with slightly different frequencies meets this requirement. It concerns the addition of two sinusoids. In this context of functions, the independent variable t (time) must appear explicitly and is displayed by a special icon. Because the sine function has not yet been introduced to the students in mathematics class at this stage of the curriculum, it is introduced as ‘some mathematical function’ of time, frequency, and amplitude, and it is mainly represented as a graph. First, students explore this function graphically, using a simulation driven by the model shown in Fig. 5. This figure is also used to introduce auxiliary variables and to explain the key feature of connectors.

Fig. 5

Graphical model for a sine function u, in which connectors indicates that t, f, and A occur in the formula defining u. The rectangle is the icon for the independent variable

Hereafter, students practise with the role of connectors in exercises on problems of variable fields inside and outside physics (variable examples principle). In these exercises, also a few stock and flow variables appear, but only as defining, respectively, defined variables. Figure 6 is used to explain dependencies: although f1, f2, A1, A2 and t are not explicitly part of the formula for u, they do have a direct influence on u. This means that u depends on these five variables. Finally, students must search and enter a suitable formula for u in terms of u1 and u2.

Fig. 6

u must be defined as the addition of u1 and u2

Design of the Module Force and Movement

In the module Force and Movement, the subsystem based on difference equations and formed by stock-flow diagrams (subsystem 2) is introduced formally, and finally, both subsystems are combined. As phenomenon to be modelled, we chose falling with air resistance. This type of movement offers several advantages for our goals. The mathematical model contains several difference equations that can be used for clarifying the relationship between difference equations and stock-flow diagrams. The combination of the difference equations Δx = v·Δt, and Δv = a·Δt is used to show that the type of a variable is not an intrinsic property, but can be manifold, depending on its roles in the equations. In addition, Newton’s second law, which was introduced to our students as the difference equation Δv = (F _net /m)·Δt, cannot be translated directly into a stock-flow diagram because it does not contain a proper flow variable. This can be solved by introducing a ‘dummy flow variable’ a = F _net /m. Doing so, the concept of flow variable is clarified. Finally, the formula for air resistance F _air  = k·v ² complicates the mathematics to such a degree that a computer model is needed. In small steps and guided by the instructional materials, students are asked to design this model. Pen-and-paper exercises have been added in which students practise all aspects discussed in “Interpretation of Diagrams” section. The exercise shown in Fig. 4 is explicitly used to introduce the distinction between the two graphical subsystems.

Classroom Setting

The classroom experiments were carried out in a Dutch secondary Montessori school. The module Sound was studied by 82 students divided over three third year classes (age, 14–15 years). In two of these classes, the first author of this paper was the teacher. A third class was taught by an experienced teacher who had also participated as a teacher at an earlier stage of the project. Because of cancelled lessons, this class did not finish the other module, Force and Movement. Therefore, the research related to this module was only carried out in the other two classes (57 students). Classes consisted of mixed groups of students, preparing for senior general secondary education and pre-university education. Near the end of the third year of secondary education, all students in the Netherlands must have chosen a study profile, i.e. a rather fixed combination of examination subjects for upper secondary level. Motivation of students not applying for a science profile often declines. At this Montessori school, it leads to a diminishing number of students participating in the tests towards the end of the year. As a consequence, in the test results of Force and Movement, there is probably a bias towards hard working and motivated students. Presumably, the aptitude for science of this group of students is more comparable with students who have chosen a science profile with average third year lower secondary groups.

Students are to some extent allowed to work at their own pace and to postpone a test until they are ready for it. They are allowed to redo a test if they fail the first time. Test results of the first attempts are used for research purposes, for pragmatic reasons and to avoid a bias towards good results.

We received 67 final tests of Sound and 27 final tests of Force and Movement. There were more than 3 months between these final tests.

Test Question of the Module Sound

The final test of the module Sound contains one question, henceforth labelled S, for probing students’ understanding of structures formed by connectors. There are two versions of the test question. In version 1, the context situation is the change of the number of customers of a shop as a result of investments in advertising (Fig. 7). In version 2, another model is used with a similar structure. Because we have not found clear differences in the characteristics of students’ answers to the two versions, we combine the results of the two versions in our analysis and use version 1 for exemplifying outcomes. This test consists of five subquestions, labelled Sa to Se, and is shown in Table 2. The purposes of the subquestions are outlined below.

Fig. 7

Diagram from the graphical model of test version 1, on the growth of the number of customers of a shop as a result of investments in advertising

Table 2 Subquestions from version 1 of the final test of the module Sound

Sa and Sb: On the Key Feature of Connectors

These subquestions address students’ abilities to determine the defining variables for a variable from the connectors in the diagram. In Sa1 and Sb1, students determine the types of variables in the diagram. The variables in Sa1 and Sb1 are defined by a formula and by a number (i.e. a constant), respectively. In case students have chosen for ‘formula’, the defining variables must be determined. In order to investigate whether students regard auxiliary variables as different from stocks with respect to connectors, both an auxiliary variable and a stock variable are part of the formula in Sa1. A student is considered to understand the key feature of connectors if (s)he correctly answers both Sa2 and Sb2. All subsequent’ answers are analysed in relation to the answers to Sa2 and Sb2.

Sc and Sd: On Dependencies of Secondarily Linked Variables

These subquestions probe the students’ understanding of dependencies as indicated by the structures of connectors and icons. In both subquestions, the chain of connectors ends in the same flow variable. The starting point is different: in Sc, it is an auxiliary variable and in Sd, it is a stock variable, the change of which is determined by the flow variable. By comparing students’ answers to these two subquestions, we want to study effects of mixing the subsystems.

Se: On Formula-based Construction of Structures of Connectors and Auxiliary Variables

In this subquestion students are asked to incorporate a new direct relation into the graphical model. The essential part of the correct answer is shown in Fig. 8.

Fig. 8

Correct extension of the diagram in subquestion Se

Outcomes of the Test Question of the Module Sound

Sa and Sb, on the Key Feature of Connectors

The number of correct answers to subquestions Sa and Sb are listed in Table 3. It shows that half of the students correctly answered all four subquestions and that about 61 % understood the key feature of connectors. Some students clearly based their answers on the diagram (by statements such as “because there are two auxiliary variable arrows pointing to the small sphere for Revenue”), but for many answers, it is not possible to distinguish whether the students based their answer on the diagram or on the realistic context situation. We found eight indications (12 % of all students) of the relation inversion conception described in “Interpretation of Diagrams” section. These students probably thought that a defined variable could also appear in the definition of one of its defining variables; some students even answered that a defined variable should be subtracted from a defining variable. The relation inversion conception did not dominate students’ thinking, though: part of these students expressed some doubt and no student made this error in both questions. Almost no students treated a defining stock differently than they treated a defining auxiliary variable.

Table 3 Results of 67 students on subquestions Sa and Sb

Sc and Sd, on Dependencies of Secondarily Linked Variables

In Table 4, students’ answers to Sc are compared with their understanding of the key feature of connectors, probed by Sa2 and Sb2. Three quarters of the students correctly answered this subquestion, even though they had practised determination of dependencies only once. Some students even spontaneously detected the feedback-loop: “It is a cycle”. But Table 4 also reveals that an important minority of this group of students did not base their answer on a fully correct conception of the relation between connectors and formulas. Some of these students showed signs of a relation inversion conception, others apparently thought that variables in graphical computer models could be defined by means of numbers only. These misconceptions do not contradict the dependency in question Sc (at least not from students’ point of view). Half of the incorrect answers could be attributed to the misconception that variables only depend on variables to which they are directly linked.

Table 4 Students’ answers to subquestion Sc, compared to their answers to Sa2 and Sb2

With respect to their explanations, some of the students clearly pointed out how the model variables are connected by means of the structure of connectors and auxiliary variables (“..because they are connected by means of arrows”), but other explanations seem to be based on an interpretation of the realistic context: “Because of the revenue per customer, they earn more and can do more advertising”.

In Table 5, students’ answers to subquestion Sd are compared with their answers to Sc. We infer that, with respect to dependencies through secondarily linked variables, the majority of students did not treat the stock variable in Sd different from the auxiliary variable in Sc. Almost all explanations to affirmative answers are correct, although in Sd, part of the explanations are based on an understanding of the realistic context. Students’ answers and argumentations show that the presence of a stock-flow diagram in the test question has been confusing for approximately a quarter of the students. There are two ways the stock-flow diagram may have influenced the students:

• 1. The stock-flow diagram, or the corresponding realistic situation, may have dominated the students’ perception. The perception that the stock depends on the flow and not vice-versa can clearly be found in the explanations of some of the students who considered the flow as depending on the auxiliary variable (in Sc), but not on the stock variable (in Sd). This student referred to the direction of the flow Customers_growth in Figure 7: “No, Customers_growth goes to Customers”. In other words, Customers_growth has influence on Customers (and not vice versa). It is possible that more students held this opinion; the others gave no clear explanation. Inspection of the students’ answers to question Sa2, in which the stock is a defining variable, reveals that this perception is not caused by an incorrect understanding of the key feature of connectors.

• 2. Four affirmative answers to question Sd were based on inversion of the stock-flow relationship.

Table 5 Students’ answers to Sd compared to their answers to Sc

Se, on Formula-Based Construction of Structures of Connectors and Auxiliary Variables

In Table 6, students’ answers to subquestion Se are compared with their answers to the subquestions probing the understanding of the key feature of connectors. Question Se is answered correctly by almost half of all students, but a quarter of these correct answers are most likely based on an incorrect understanding of connectors. Most students answers from this quarter show signs of the relation inversion conception or of the conception that in graphical models only numbers are used to define variables.

Table 6 Students’ answers to subquestion Se compared to their answers to Sa2 and Sb2

Other students correctly translated the direct relation into a model structure, but added an extra connector. The most frequent addition (Fig. 9) may have been based on students’ interpretation of the corresponding realistic situation, in which it would make sense to calculate part_for_advertising from Revenue. Apparently, these students understood how connectors correspond to direct relations, but they also added a reality-based element. Some students gave the incorrect answer shown in Fig. 10. Here, the diagram seems to be viewed as a flow chart for the money.

Fig. 9

A frequent answer to subquestion Se. This diagram is in accordance with the direct relation that must be added, but also an extra connector is drawn

Fig. 10

Example of an incorrect answer to subquestion Se from test version 1

Summary of the Outcomes of the Test Question from the Module Sound

Approximately half of the students correctly answered all questions about structures formed by connectors. Per item, the scores were somewhat better. Apparently, many students had a basic understanding of the key feature of connectors. Also, many students correctly understood dependencies. Only 10 % of all students considered secondarily linked variables as not dependent on each other. Only a minority of students had been troubled by the presence of a stock-flow diagram in a chain of connectors. Constructing (part of) a model was the most difficult task. The outcomes of the subquestions also show that students arrived at correct conclusions with respect to model structure without having a complete understanding of the underlying concepts, using alternative conceptions:

• Students used their understanding of the realistic context and did not interpret the diagram structure.

• Students holding a relation inversion conception arrived at correct answers to almost all questions.

• The misconception by students that all model variables may only be defined by numbers did not lead them to incorrect conclusions regarding dependencies.

Test Questions from the Module Force and Movement

Three test questions, F1, F2, and F3, have been designed. F1 probes the interpretation of a diagram. Its structure is similar to the test question in Sound. F2 and F3 are construction questions. Purposes of the questions are outlined in below subsections.

Test Question F1, on the Interpretation of Diagrams

Question F1 consists of four subquestions. In subquestions F1a to F1c, students must derive information about the definitions of three model variables from the diagram in Fig. 11. Students’ understanding of dependencies is investigated in F1d.

Fig. 11

Diagram from test question F1. Subquestions are about the encircled model variables

Subquestions F1a, F1b, and F1c, on the Definitions of Model Variables

In F1a to F1c the focus is on students’ interpretations of stock-flow diagrams and on effects of mixing stock-flow diagrams and structures of connectors.

• In F1a, the defined variable is an outflow of a stock-flow diagram (H_decrease in Fig. 11). This outflow must be defined by a direct relation consisting of an auxiliary variable, a stock variable from a second stock-flow diagram, and the stock-variable from its own stock-flow diagram. This combination has been chosen to find out to what extent students are confused by the stock-flow diagram which the outflow is part of.

• In F1b, the defined variable is an auxiliary variable that is defined by a number.

• In F1c, the defined variable is the stock from the stock-flow diagram used in F1a (Hares in Fig. 11). The question is whether students realise that in the formula defining the change of the stock only both its flows and the time step can appear. This stock is also directly linked to three other model variables. A question for research is to what extent students are confused by the presence of these other variables.

Subquestion F1d, on the Dependency of a Stock on its Outflow

In subquestion F1d, students are asked whether the stock Foxes in Fig. 11 depends on the constant auxiliary variable F_mortality_factor and to explain their answers. These variables are secondarily linked, with the outflow of the stock as an intermediate variable. This subquestion has been designed to investigate:

• 1. Whether students consider a stock variable as dependent on its outflow(s). The difference between F1c and F1d is that in F1c, students are asked about the definition of the stock, whereas F1d is about dependency. In F1d, the relation between stock and flow is one of two relations linking the auxiliary variable F_mortality_factor secondarily to the stock Foxes. Students’ explanations are also analysed with respect to the misconception that variables do not depend on each other in case they are not directly linked.

• 2. To what extent students base their answer on the graphical model, and to what extent their understanding of the realistic context is involved.

Test Question F2, on Formula-based Construction of Diagrams

The students’ ability to construct a diagram for a given system of formulas is investigated. The formulas are a difference equation containing one flow variable and a direct relation defining this flow, in the context of electrical circuits.

Test Question F3, on the Construction of a Graphical Model for a Realistic Situation

This question is designed to probe to what extent students are able to construct simple versions of both types of formulas (direct relations and difference equations) for a given context situation, and to investigate how their ability to construct these formulas relates to their ability to construct the diagram for the realistic situation. The context is forestry. The correct difference equation contains two flow variables, of which the inflow is a constant and the outflow depends on the stock variable. Students are asked to construct this difference equation (F3a), the direct relation for one of the flow variables (F3b), and the diagram (F3c).

Outcomes of the Test Questions of the Module Force and Movement

F1, on the Interpretation of Diagrams

F1a to F1c, on the Definitions of Model Variables

To subquestion F1b, about the model variable that must be defined by a number only, 89 % of all students’ answers are correct. Occasionally, signs of a relation inversion conception can be detected.

Results on questions F1a and F1c are shown in Tables 7 and 8, respectively. Only a very small number of students incorrectly thought that the model variables in these questions are defined by numbers, but many of the other students had problems with the designation of these formulas. Misnaming occurred more often in the case of a formula defining a flow than in the case of a formula defining a stock. Apparently, students are tempted to call any formula that involves a flow or a stock a difference equation. But in graphical modelling, flows can only be defined by direct relations or numbers, and stocks are always defined by means of difference equations.

Table 7 Student answers to test question F1a

Table 8 Student answers to test question F1c

Table 7 shows that a majority (70 %) of the students understand the key feature of connectors in spite of problems with designation. Table 8 shows that understanding the relation structure of stock-flow diagrams is more problematic: merely 41 % of all students correctly mentioned both flow variables as being part of the definition for the stock variable in test question F1c. An important error is mentioning the inflow, but not the outflow. We thought of two reasons for the alternative conception that a stock is not influenced by its outflow(s). The first reason is diagram-based: students may be confused by the similarity between flow icons and connectors, because both are represented by arrows. The second reason may be that students think that an outflow does not have an influence on a stock. In answers to other questions, students often only discuss increases of variables and rarely mention decreases. If this is what students really think, then their answers on test questions F1d and F3 should also indicate this, however, we only found one indication of a student thinking that outflow does not have an influence on a stock.

Furthermore, no students mentioned time or a time step as part of the formula defining the stock. Other student errors in test question F1c were the reversion of the connectors linked to the stock variable and confusion of ‘dependency’ with ‘being part of a formula’ for constants that are directly connected to some part of the stock-flow diagram.

F1d, on the Dependency of a Stock on its Outflow

The students’ answers to subquestion F1d have been analysed with respect to two aspects, namely, the relations identified in the students’ explanations and the nature of the explanations. This has been done for both correct and incorrect answers. The identified relations are labelled in Fig. 12. A correct explanation refers to the existence of relations 1 and 2 and is based on elements of the diagram and/or on formulas. The numerical outcomes of this analysis are shown in Table 9.

Fig. 12

The four possible relations that may be discussed by the students in F1d are indicated by numbers in rectangles. Relations 1 and 3 are clearly indicated by connectors. Relation 2 is the dependency of the stock Foxes on the outflow F_mortality (because F_mortality is part of the Δ-formula defining the change of Foxes). Relation 4 does not exist but it is here represented by a dotted arrow. Students may use the absence of this direct relation as an argument

Table 9 Students’ answers and students’ explanations of their answers to test question F1d

Less than half of all students drew the correct conclusion that the stock variable Foxes depends on the auxiliary variable F_mortality_factor, to which it is secondarily linked. Students who arrived at the correct conclusion all referred to the relevant relations, but half of these students referred in their explanation to the realistic context, by statements such as: “When foxes die, the mortality factor is important. If it is very bad, more foxes will die and the stock variable ‘Foxes’ will decrease”. As this example shows, these students did not necessarily ignore the diagram, but they interpreted the diagram in terms connected to reality. The other half of the students referred to elements of the diagrams, or to calculations and/or formulas.

The students who arrived at an incorrect answer based their explanation on elements of the model structure, but their argumentation is based on an incorrect understanding of these elements. This group contains almost all students according to whom the outflow is not part of the definition of the stock in subquestion F1c. Relation 2 is not mentioned or even explicitly denied by these students. The lack of relation 4 (between these secondarily linked variables) is also used as an argument; for 11 % of the students, this was the only argument.

F2, on Formula-based Construction of Diagrams

As shown in Table 10, the majority of students correctly constructed the simple diagram shown in Fig. 13, based on a difference equation and on a direct relation that must define the flow. All students who answered F1d correctly, and most students who in F1c only mentioned the inflow, correctly answered question F2. In case students had problems with the construction of the diagram, this generally was with respect to the stock-flow diagram, and not with respect to connectors. Finally, in all answers that are not completely correct, an icon for time or for the time step Δt has been added.

Table 10 Students’ results on test question F2

Fig. 13

Correct diagram to the formulas ΔQ = I·Δt and I = U/R in test question F2

F3, on the Construction of a Graphical Model for a Realistic Situation

In the context of forestry, in this section, the symbol B is used for the stock variable (the number of trees in the forest), the symbol A for the constant inflow (the number trees planted each year) and the symbol K for the outflow (the number of trees that are cut each year).

Construction of Difference Equations and Corresponding Stock-Flow Diagrams

In Table 11, the main students’ results regarding the construction of the difference equation in question F3a and the related stock-flow diagram in question F3c are shown. This table reveals that students who constructed a correct difference equation also constructed a correct stock-flow diagram, although some errors were made concerning connectors. Below, we focus on the students who constructed an incorrect difference equation.

Table 11 Quality of the constructed difference equation in question F3a versus the quality of the constructed stock-flow diagram in question F3c

Of all students who did not construct a correct difference equation, 11 students nevertheless built a correct stock-flow diagram, although none of them drew a completely correct standard diagram, consisting of an inflow, an outflow, and one correct connector. Two students drew a correct alternative solution, the others made errors with respect to connectors. All students who were of the opinion that a stock does not automatically depend on its outflow did not draw a completely correct standard diagram. Apparently, this alternative conception has influenced the construction of the diagrams, but in different ways. Analysis of the errors in the difference equations gives more insight. Some errors do not necessarily hinder the construction of the stock-flow diagram. Such errors are the exclusion of Δt from the difference equation and the incorporation of an initial value into the difference equation. Remaining questions are how to explain why most of these students made errors with respect to connectors, and how to explain that students who left out a variable for the outflow or the inflow from the difference equation still could construct a correct stock-flow diagram or alternative diagram. Plausible explanations are found by considering consequences of the conception that the stock does not depend on the outflow. In case the difference equation in question F3a contains the variable for the outflow, from this point of view, there is a conflict that can be solved in two ways:

• 1. By constructing an alternative solution without using an icon for the outflow, as in Fig. 14

  Fig. 14

  

  Answer to question F3c of a student who in “F1a to F1c, on the Definitions of Model Variables” section was of opinion that a stock does not depend on an outflow. The only flow in this diagram is an inflow, and what should have been an outflow (“K”) is represented by an auxiliary icon which is linked to this inflow. Apart from the fact that the connectors are not represented as arrows, this diagram is correct

• 2. Incorrectly, by adding a connector from the outflow to the stock, which should indicate that the stock depends on the outflow, as in Fig. 15

  Fig. 15

  

  Diagram to question F3c of a student who in “F1a to F1c, on the Definitions of Model Variables” section was of opinion that a stock does not automatically depend on an outflow. Erroneously, a superfluous (and incorrect) connector is drawn from the outflow (“K”) to the stock (“B”) to indicate that in this case the stock depends on the outflow

We found several examples of both approaches in the students’ work. Another consequence of the conception that the stock does not depend on the outflow may be that the variable for the outflow should not appear in the difference equation. We found a few examples of such difference equations created by students. These students drew an outflow but also added a connector to the diagram in a way similar to Fig. 15, or they did not draw a diagram at all.

Most of the students who did not construct a correct stock-flow diagram in question F3 also did not construct a correct stock-flow diagram in question F2. A common error was the addition of an icon to the diagram for the time step in question F2. Further analysis with respect the time step Δt reveals the following:

• When asked to construct a diagram for a given difference equation in question F2, 30 % of all students (incorrectly) added an icon for Δt.

• When asked to construct the difference equation for a realistic situation in question F3a, 33 % of all students (incorrectly) left out Δt of the equation.

• But when asked to construct a diagram for a realistic situation in question F3c, 85 % of all students constructed a diagram without an icon for time or time step. This seems to be a slightly better result. Indeed, one of the arguments in favour of graphical modelling in education is that an understanding of difference equations can be circumvented. In addition, the fact that in question F1c no student mentions time as one of the variables that appear in the formula defining a stock-variable, may mean that students correctly do not consider time as a causal agent, as is suggested by Kurtz dos Santos and Ogborn (1994). But our results indicate that a reality based construction of a stock-flow diagram may well conceal an incorrect understanding of the role of time and time step. Students may even not think of the role of time at all.

Construction of Direct Relations and Corresponding use of Connectors

In question F3b, the formula for the number of trees cut each year (K) must be constructed by the students. It is given to be 20 % of the number of trees (B). The correct answer is a simple direct relation involving B.

Main outcomes with respect to the construction of this direct relation and the related structure of connectors in the diagram in question F3c are shown in Table 12. Half of the constructed direct relations are completely correct. When minor errors, such as calculation errors, and errors concerning a less adequate conception of variable and formula are ignored, we can state that 78 % of all students realised that K is a function of B.

Table 12 Consistency of the constructed direct relation and the constructed diagram in question F3

Approximately, one third of the students drew a diagram in which the structure of connectors can be considered as consistent with their answer to question F3b. This number of consistent diagrams is far less than in question F2. Apparently, question F3, in which the stock-flow diagram and the structure of connectors are more closely intertwined, is more difficult. As outlined in the section “Construction of Difference Equations and Corresponding Stock-Flow Diagrams”, an important cause for incorrect structures of connectors is the fact that K in question F3 is an outflow. The influence of the realistic context situation seems to be a cause for another error: 19 % of the students drew a connector from the stock B to the inflow, although this inflow was given to be constant. An alternative explanation for this incorrect connector is that students merely have copied elements from the diagram of question F1.

Summary of the Outcomes of the Test Questions from the Module Force and Movement

Results confirm that most of the students understand the subsystem formed by connectors and are not troubled by the effects due to mixing of this subsystem with stock-flow diagrams. A vast majority can correctly determine defining variables and can construct simple diagrams as far as it concerns connectors. Only a few students’ answers show signs of relation inversion.

Students have more difficulty with the relation between stock-flow diagrams and difference equations. There are two main misconceptions. The first is that students do not sufficiently understand the role of the time step. As a consequence, they forget that the time step is part of the difference equation or they add an icon for time or time step when a difference equation must be converted to a stock-flow diagram. The second alternative conception, held by a quarter of the students, is that a stock variable does not depend on its outflow. A consequence is that students, trying to deal with this conception, make errors with connectors. The main reason for this alternative conception is the fact that flows and connectors both are represented as arrows. Students insufficiently appreciate that both types of arrows have completely different meanings. This can be considered as an effect of mixing the subsystems. Another effect of mixing may be that students are tempted to call any formula that involves a flow a difference equation. Finally, students have problems with the role of initial values. In spite of these problems, results are promising.

From students’ explanations to their answers, we conclude that students who involved the realistic context in their explanations more often correctly identified dependencies between variables than students who based their answers only on the structure of the diagram. This makes it difficult to distinguish what students are understanding when they correctly answer questions concerning a model: the diagram or reality. All students who use mathematics-based explanations gave correct answers, but only a few used this type of explanation.

Conclusions and Discussion

The research question addresses students’ understanding of relations structures in graphical modelling after relation-based instruction. In this section, we summarise our answer with regard to three aspects: students’ understanding of structures formed by connectors, students’ understanding of stock-flow diagrams, and the influence of the realistic situation on students’ reasoning and understanding. We position our instructional design and research findings with respect to approaches and outcomes reported in literature. We briefly discuss why traditional approaches seemingly are successful when students work with models but often fail when students must construct models themselves. We also discuss the validity of the research outcomes and make recommendations for future design.

Students’ Understanding

Students’ Understanding of Structures Formed by Connectors and Direct Relations

Traditional approaches to graphical modelling often use causal reasoning to describe model relations and dependency of model variables (cf., Groesser 2012a). Our research shows that a relation approach is an alternative effective way to teach young students to interpret the structures of graphical models formed by connectors and construct such structures. We found that, in this approach, dependency visualised by chains of connectors can be understood by many students too, although some of them arrive at correct conclusions by reality-based reasoning instead of (or together with) diagram-based reasoning, or by using alternative conceptions. Identified alternative conceptions are the relation inversion conception and the conception that in a model only numbers are required, and not formulas involving variables. Such misconceptions can easily be left unnoticed in an approach in which little attention is paid to the mathematical meaning of the relations. Only 10 % of all participants in this study still think that there is no dependency between secondarily linked variables. The effects on students’ understanding also hold when a stock is incorporated at the beginning of a chain of connectors and a flow at the end. Only occasionally, students incorrectly ignored such a chain in favour of the one-step relation between a flow and corresponding stock.

Students’ Understanding of Stock-Flow Diagrams and Difference Equations

In traditional approaches to graphical modelling, model conceptualisation usually starts with the determination of stocks and flows straight from the realistic situation. Difference equations are seldom introduced. This is a fundamental difference with a relation approach, in which model conceptualisation consists of two steps: the determination of the required equations for the realistic situation, and equation-based construction of the diagrams. We claim that a relation approach circumvents some of the well-known difficulties with traditional stock-flow approaches listed in the section “Approaches to Graphical Modelling”. These are the difficulties with distinguishing stocks and flows, and obstacles related to the stock-flow metaphor. This does not mean that all known problems with graphical modelling have been solved yet, but these problems do not stay underneath the radar and can be explicitly addressed in instruction.

In a relation approach, in the context of physics, the determination of the difference equations usually is not problematic because many of these difference equations are well-known and are part of the physics curriculum anyway. They form an appropriate starting point for the majority of students. Results with respect to equation-based construction of stock-flow diagrams are promising. Moreover, our research indicates that many of these young students can also construct a simple difference equation for a new context situation. But there still remains some student difficulties that must be solved with equation-based construction of stock-flow diagrams.

Two of these difficulties can be considered a consequence of the fact that the difference equations are not clearly visualised in the stock-flow diagrams. The first difficulty is that some of the students are not fully aware of the presence of the time step Δt in the difference equation. This lack of awareness can be left unnoticed in cases where students are asked to construct a stock-flow diagram for a realistic situation without creating a difference equation, as is common practice in traditional approaches. The second difficulty concerns the role of the initial value with respect to the difference equation. This role is not made clear by the diagrams or the dialog window for defining a stock. The fact that for a stock only the number for the initial value must be entered may even contribute to the misconception that a stock is not defined by a formula at all (Van Buuren et al. 2012). One goal of the diagrams is communicating the model to people with less mathematics education (cf., Lane 2008), where an understanding of difference equations would not be required. This may be true in case these people only have to interpret the graphical models, or listen to a modeller explaining the model, but the diagrams do not sufficiently communicate the equations for (some of the) novice modellers who are expected to learn to construct models themselves.

A third difficulty is the misconception that a stock does not depend on its outflows. The main reason for this misconception is that both flows and connectors are represented as arrows. This conception, of considering the arrows of the flows in the same way as the connectors, has been explicitly addressed in the instructional materials (Refer to “Overview of the Relation Approach in de Modules Sound and Force and Movement”) but, apparently, is not easily changed.

The question is how these difficulties can be overcome. Segmenting the learning material may have been effective for understanding the subsystem formed by connectors alone, but is apparently not sufficient for teaching students how to understand entire diagrams. From the perspective of cognitive load theory, the diagrams must be considered as high element interactivity material (cf., Sweller 1994). The links between the elements of the diagrams and their mathematical counterparts are deliberately not visualised in system dynamics tools for the purpose of allowing modellers to focus on the main structures of the model. Such links may be redundant for experienced modellers, but for novice learners they are not, as Groesser (2012b) points out and as our results confirm. By not presenting the graphical elements and mathematical counterparts simultaneously, extraneous load is increased (split-attention effect). We think that this extraneous load can be diminished by improving the design of the modelling learning environment.

Influence of the Realistic Situation on Students’ Reasoning and Understanding

In the section “Outcomes of the Test Question of the Module Sound”, part of the explanations of the students of their answers seem to be based on reality instead of on the diagram. In “F1d, on the Dependency of a Stock on its Outflow”, it is shown that students who referred to reality in their explanations more often arrived at the correct conclusion about the dependencies of variables than students who based their explanation only on their (incorrect) understanding of the diagram. Apparently, there are two ways for students to understand a model for a real situation: one is based on the diagram structure, the other on the real situation. This makes it difficult to distinguish what students actually understand when they answer questions about the model correctly: the model or the real situation. In the cases we have investigated, we suspect that the students have used their knowledge of reality and some logic to correct their interpretation of the diagrams. We noted that some of the students constructed a stock-flow diagram directly from reality more successfully than they constructed a difference equation from reality or a stock-flow diagram from a difference equation. However, this seemingly correct reality based construction of a stock-flow diagram may conceal an incorrect understanding of graphical models. This phenomenon may help to explain findings of research in system dynamics modelling hitherto, that modelling activities seem to be successful only until students are asked to construct parts of graphical models without assistance (Westra 2008; Van Borkulo 2009). Communication of the causal assumptions and the main features of the mathematical model are supposed to be the main purposes of the diagrams. The diagrams apparently are helpful for students to organise their thoughts on reality in a way more or less similar to concept maps. But this strength of the diagrams is at the same time a weakness. Students do not need to fully understand the diagrams for interpreting them. The icons can be interpreted in several ways. An ‘arrow’ can have at least three different meanings: (1) indicating relationship; (2) representing a flow variable, in which case the arrow gives information on the sign of the flow variable in the difference equation for the stock; or, (3) incorrectly, suggesting a direction on a flow chart. Because these arrows are not adequately labelled, they can easily be misinterpreted by novice modellers. This shows some resemblance with concept mapping. Hilbert and Renkl (2008) found that the concept-map characteristics that relate most to learning outcomes are the number of correctly labelled and unlabeled links. Here, it also seems to be important for learning that the type of relationship between the concepts is clearly specified. Other examples of student difficulties resulting from unclear arrow symbolism have been identified by Du Plessis et al. (2003) for biology diagrams.

Only when students are asked to construct diagrams, or to debug an existing incorrect diagram, does it becomes clear whether students really understand the graphical models. Quoting one of the students of Westra (p.178): “I understand a model when it is explained, but I am not able to build it myself”.

It must be emphasised that this holds for novice modellers. For experienced modellers, who already have some understanding of the equations, the difficulties we have established in this paper can easily be overcome. It must be noticed that some of the difficulties we have found will not necessarily be present when another approach is used, but we suspect that, with another approach, other misconceptions will arise. It is plausible that these other misconceptions can also stay concealed because students have the powerful alternative strategy at their disposal of using their knowledge of reality to interpret the diagrams, until they are asked to construct graphical models. An advantage of our relation approach is that it offers a clear strategy for construction of graphical models which starts with the equations. In particular, this advantage holds for disciplines such as physics, in which formulas are commonly used.

In summary, three categories of difficulties with graphical models can be distinguished:

• 1. Reality-based interpretation can conceal that graphical diagrams are not understood.

• 2. Mathematical aspects are insufficiently communicated by the diagrams.

• 3. Icons in the diagrams can be misleading.

Validity of the Outcomes

As indicated, a statistical analysis of this explorative research is of limited value. In particular, the results of the test of Force and Movement probably contain a bias towards hard-working and well-motivated students. Therefore, quantitative results can only be considered as indicative, especially when they concern positive learning effects. On the other hand, the difficulties and misconceptions that we have found can be expected to be even more frequent among a general group of students. In order to improve statistical reliability, the research must be repeated on a large scale with many more student groups at secondary schools. This was beyond the scope of this research.

For reasons of ecological validity, our qualitative results are derived from students’ answers to the final tests of the modules in a rather deductive way. Teachers’ observations and impromptu interviews are in agreement with this deduction. The conclusions are plausible, but separate in-depth interviews may strengthen them. This can be done in follow-up research.

Recommendations for Future Instructional Design

The relation between stock-flow diagrams and difference equations should be made clear for novice modellers. This might be done by showing the difference equation in the dialog window for the stock. Another possibility is to show the students how the diagram can be translated into a text-based model at an early stage of the modelling learning path. In Coach 6, an option is available for this translation. But, probably, the most effective is the implementation of an option into the modelling software to permanently show all equations to the relevant icons, because this would diminish the split-attention effect and therefore would reduce extraneous cognitive load for novice modellers. This is possible in Simulation Insight (see, for example, Rogers 2008), but in this modelling environment, all variables are represented by rectangular icons, linked by means of connectors. The type of a formula is not expressed by the structure of the icons.

We also recommend adaptation of the icons for the flow variables. There exist several implementations of system dynamics based graphical modelling. According to Lane (2000), there are at least three or four widely used symbols for flow variables, all taking the form of valve or regulator symbols on the flows, which are indicated by double or thick arrows. In the modelling environment Modus, arrows for both the inflows and outflows are oriented from the flow variable to the stock (Klieme and Maichle 1991). This may prevent the conception that an outflow does not influence the stock, but does not yet clarify the relation between difference equations and stock-flow diagrams. In our opinion, a suitable flow icon should fulfil two conditions:

• 1. The only difference between an auxiliary variable and a flow variable is that the latter is part of a difference equation. In all other respects, flow variables and auxiliary variables are treated the same in the diagrams. Therefore, we recommend to use more similar icons. In Fig. 16, this similarity is already clearer.

  Fig. 16

  

  Stock-flow diagram in STELLA

• 2. Icons for flows should differ more from the icons for connectors. A question is whether flow icons should be arrow-like. From the perspective of a relation approach, a tube-like symbol in which flow directions are indicated by means of a + or − sign, as sketched in Fig. 17, or by means of a word (in or out), might be an alternative.

  Fig. 17

  

  Sketch for a stock-flow diagram using alternative icons for the flows