Abstract
Instruction often spans multiple phases (e.g., phases of discovery learning, instructional explanations, practice) with different learning goals and different pedagogies. For any combination of multiple phases, we use the term composite instructional design (CID). To understand the mechanisms underlying composite instructional designs, we propose a framework that links three levels (knowledge, learning, instruction) across multiple phases: Its core element is the specification of learning mechanisms that explain how intermediate knowledge (i.e., the knowledge state between instructional phases) generated by the learning processes of one phase impacts the learning processes of a following phase. The CID framework serves as a basis for conducting research on composite instructional designs based on a cognitive analysis, which we exemplify by discussing existing research in light of the framework. We discuss how the CID framework supports understanding of the effects of composite instructional designs beyond the individual effects of the single phases through an analysis of effects on intermediate knowledge (i.e., the knowledge state resulting from a first instructional phase) and how it alters the learning processes initiated by the instructional design of a second phase. We also aim to illustrate how CID can help resolve contradictory findings of prior studies (e.g., studies that did or did not find beneficial effects of problem solving prior to instruction). Methodologically, we highlight the challenge of altering one learning mechanism at a time as experimental variations on the instructional design level often affect multiple learning processes across phases.
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Avoid common mistakes on your manuscript.
Introduction
Instruction – both in research and everyday classroom practice – often spans multiple phases with different learning goals and different pedagogical approaches (Kunter, 2005; Prediger et al., 2021; Stigler et al., 1999). For example, phases of discovery learning and phases of instructional explanations can be systematically combined. To describe any combination of instructional approaches in multiple-phase instructional designs, we introduce the term “composite instructional designs”. Scholars and practitioners alike argue that composite instructional designs can draw on the advantages of each of their components and, thus, have a high likelihood to improve learning (e.g., Kalyuga & Singh, 2016; Loibl & Leuders, 2018). In an in-depth review of the existing evidence on inquiry learning, for instance, de Jong et al. (2023) highlighted that often the best approach to supporting student learning is to combine inquiry and direct instruction. However, research on composite instructional designs has yielded inconclusive findings (e.g., Glogger-Frey et al., 2015 vs. 2017; Hartmann et al., 2022 vs. Kapur, 2014), which cannot be explained by the learning mechanisms that have been theoretically discussed within this line of research so far. The present paper aims to develop a framework for the cognitive analysis of composite instructional designs (CID) to address these issues.
To understand the mechanisms underlying the advantages of any type of instructional design, Koedinger et al. (2012) proposed a so-called knowledge – learning – instruction (KLI) framework. The authors argued that relating elements on the three levels of (K) knowledge components, (L) learning processes, and (I) instructional design enables deeper insight into how instructional design elements initiate learning processes and result in better learning outcomes.
To illustrate this approach, let us consider an example of a design with only one single phase, which aims to explain learning outcomes by means of the underlying learning mechanisms, that is, the theoretical assumptions on the causal relation between knowledge components and learning processes initiated by instructional design elements: The theory on instructional explanations by Wittwer and Renkl (2008) links the design of instructional explanations to students’ knowledge and the learning processes in the following way: On the instructional design level, instructional explanations need to be tailored to the students’ prior knowledge. Such explanations allow the learners to construct a coherent mental representation provided that they actively engage with the given information. Nückles and colleagues (2006) demonstrated that students who received tailored instructional explanations (instruction level) generated more elaborative self-explanations (learning process level), and thereby reached a deeper understanding (knowledge level) than students who received instructional explanations that were not tailored to their prior knowledge.
Due to their inherent complexity, research on composite instructional designs, that is, designs with multiple phases based on different pedagogies, is much less frequent than research on single-phase designs. Such research often targets broader questions concerning the learning outcomes of the design as a whole, such as: Does the composite instructional design in question foster learning outcomes more than the single phases? Which order of the phases has the largest effect on learning outcomes? These questions focus on the effect of variations of the instructional design level on the knowledge level, but research on composite instructional designs addressing these questions only rarely analyzes the learning process level, even though this is essential to yield explanations for the effects on learning outcomes.
The literature on composite instructional designs that explicitly discusses learning processes focuses specifically on processes that are crucial for the constituent phases. This focus is indicated by terms like “preparation for future learning” (Schwartz & Martin, 2004), “problem solving prior to instruction (PS-I)” (Loibl et al., 2017), or “advance organizer” (Ausubel, 1960; Gurlitt et al., 2012). This literature theoretically reflects the levels of instructional design and learning processes by addressing questions such as: Which learning processes are initiated by the instructional design of each phase? Do the learning processes initiated by the instructional design of one phase prepare for learning in a subsequent phase? Do the learning processes initiated by the instructional design of the second phase build on and complement the first phase?
Unfortunately, research on composite instructional designs has frequently yielded inconclusive findings. For instance, results on the effects of problem solving vs. worked solutions as preparation for subsequent instruction are contradictory (Schwartz et al., 2011; Glogger-Frey et al., 2017, 2022; Newman & De Caro, 2019). We argue that a systematic reason for these inconclusive findings may lie in the interdependence of the learning mechanisms across the different phases (defined by relations between elements at the knowledge, learning, and instruction level). For example, working through a worked solution to a problem as compared to solving the problem in the first phase can alter the depth of knowledge acquired during this phase, which in turn also affects the learning processes in the second phase (Glogger-Frey et al., 2022). To put it more generally, changing the instructional design of the first phase alters the learning processes in this phase, resulting in different knowledge as the starting point for the subsequent phase. As the learning processes in the subsequent phase build on the knowledge resulting from the first phase, the learning processes will necessarily also unfold differently. Therefore, it is problematic to consider learning mechanisms of single phases within composite instructional designs in isolation. Even without changing the instructional design of the constituent single phases, the mere switching of the sequence of the phases, which has been a paradigm in research on composite instructional designs (e.g., in research on productive failure or PS-I: Sinha & Kapur, 2021; Loibl et al., 2017), impacts the learning mechanisms in the second phase due to the knowledge acquired during the first phase. Below, we provide detailed examples of such interdependencies of learning phases and resulting changes in learning mechanisms, and discuss why these interdependencies may be the reason for the inconclusive findings.
In this article, we develop the argument that when investigating composite instructional designs, it is beneficial to link theoretical elements from all of the three levels – the knowledge components, the learning processes, and the instructional design – to examine their interplay across the phases, and to include all three levels by posing research questions such as these: Which knowledge components – that can be built upon in the second phase – are activated or generated via learning processes triggered by the instructional design of the first phase? How do the knowledge components resulting from the first phase (and not merely the instructional design of the second phase) affect the learning processes in the second phase? Such research questions focus on the interdependencies of the phases in composite instructional designs, thus facilitating the empirical investigation of the underlying learning mechanisms.
Building on the arguments outlined above, we propose a framework that links the three levels (knowledge, learning, instruction) across multiple phases (cf. Section "The CID framework and its purpose"). This framework serves as a basis for studying composite instructional designs, which we exemplify by discussing existing research in light of our framework. We demonstrate how the framework can support our understanding of the effects of composite instructional designs beyond the individual effects of the single phases discussed so far, and how it can help resolve the seeming contradictions in empirical findings (cf. Section "Exemplary applications of the CID framework"). Finally, we provide an outlook on how the framework might be used in future research on composite instructional designs by discussing its advantages and limitations (cf. Section “Discussion”).
The CID framework and its purpose
The structure of the CID framework
We present a framework for conducting cognitive analyses of composite instructional designs (CID) that links the three levels – (K) knowledge components, (L) learning processes, and (I) instructional designs. The framework follows Koedinger and colleagues’ (2012) argument that when designing and investigating learning environments, one has to consider how these three levels relate to and build on each other in a specific type of instructional setting and for specific types of addressed knowledge components. The CID framework uses this approach to focus on the interaction between the phases of a composite instructional design. We argue that this is essential in order to thoroughly understand the theoretical underpinnings of the learning mechanisms at work. The elements of our framework and their relations are outlined in Fig. 1 for the simplest composite instructional design: a design with two phases.
The CID framework integrates the phases of a composite instructional design and the three levels – knowledge, learning, and instruction. Note that although knowledge is evolving continuously throughout the learning phases, we focus on three moments (of assessment) in our theoretical explanations: prior knowledge (K0), intermediate knowledge (i.e., knowledge state after the first phase; K1), and final knowledge (i.e., knowledge state after the second phase; K2)
When instructional designs with two (or more) phases are to be investigated and optimized with respect to their learning outcomes (see Fig. 1, I1+I2→K2), the main research interest is on the combination of the two instructional elements (I1 and I2), that is, the added benefits above and beyond the sum of their individual effects. While the individual effects of the instructional elements are usually explained by learning mechanisms initiated by each element in isolation, the composite effect of both elements together necessarily relies on what happened in consequence of I1, before the onset of I2, or, more precisely, on the knowledge acquired in the first phase (K1), which we term intermediate knowledge. Intermediate knowledge may comprise knowledge components of the target knowledge (i.e., intended final knowledge), activated prior knowledge, but also erroneous knowledge that requires refutation, knowledge about boundaries of one’s own knowledge (knowledge gap awareness, Loibl et al., 2017; Glogger-Frey et al., 2015), or procedural or strategic knowledge that supports further learning processes. We argue that any theoretical analysis that seeks to contribute to understanding a composite instructional design needs to consider effects on the learning mechanisms of the second phase (L2) resulting from the previously acquired intermediate knowledge (K1). The diagram also illustrates that the quest for theoretical explanations must include the knowledge level and cannot be limited to the learning process level or even the instruction level. Moreover, modelling the initial state (prior knowledge, K0) allows accounting for the impact of individual differences and thereby possibly explaining aptitude-treatment-interactions (Cronbach & Snow, 1977; Preacher and Sterba, 2019), that is, interactions between students’ prior knowledge (aptitude) and specific instructional designs (treatment). For instance, the different impact of sequences of instructional phases on students with low or high prior knowledge (Zhang & Sweller, 2024). Such expertise-reversal effects (Kalyuga, 2007) are explained at the level of the learning process by a decrease of element interactivity (several elements of the concept stored as one higher-level element) with an increase in prior knowledge (Sweller, 2010). Thus, modelling prior knowledge (K0) can provide explanations for divergent effects of different implementations of composite instructional designs.
Koedinger and colleagues (2012) demonstrated a useful way to specify the knowledge and learning levels: The authors conceive of learning as an ongoing process that leads to changes on the knowledge level, where multiple stages build on each other and various knowledge components are constructed or connected, such as linking a procedure with its rationale. In order to derive testable predictions, these knowledge components need to be specified for each learning topic. Knowledge components that are typically involved in learning include: associative knowledge, such as facts (e.g., a right angle has 90°) or exemplars (i.e., prototypical cases or solutions, Renkl, 2023, e.g., the side lengths (32 + 42 = 52) of a concrete right-angled triangle); procedural knowledge (i.e., non-verbal skills, Anderson & Lebiere, 1998; VanLehn, 1996, e.g., calculating the missing side in a right-angled triangle with two given sides); conceptual knowledge, such as propositional networks (Kintsch & Greeno, 1985, e.g. the Pythagorean theorem connects three sides of a right-angled triangle via a relation of squares), mental models (Gentner & Stevens, 1983), and negative knowledge (i.e., knowledge about incorrect applications of knowledge including conditions and reasons for the non-applicability, Oser et al., 1999, e.g., if a triangle has no right angle, then the theorem of Pythagoras cannot be applied). Note that a knowledge component can also be erroneous, as in the case of a misconception or a flawed procedure (Chi, 2000). The knowledge components may also influence each other during the learning process (for interactions between procedural and conceptual knowledge see Rittle-Johnson et al., 2001, 2015). Learning goals often include the ability to apply these knowledge components to new situations or problems, which is usually referred to as knowledge transfer (Barnett & Ceci, 2002).
These different knowledge components are acquired via different types of learning processes: Roughly speaking, sense-making processes foster conceptual knowledge, and refinement or fluency-building processes foster procedural knowledge or associative knowledge (Koedinger et al., 2012). The different knowledge components, in turn, influence subsequent learning processes differently. Koedinger et al. (2012) provided three examples of how different types of knowledge may support subsequent learning, without explicitly considering multiple phases in composite instructional designs: (1) Acquired learning strategies can be applied for subsequent learning; (2) Foundational skills or concepts may be transferred to new domains; (3) Fluency may free up cognitive capacity for further learning processes.
To restate this more specifically for composite instructional designs: Prerequisites for successful learning in a composite instructional design are a purposeful activation of learning processes in the first phase (Gurlitt & Renkl, 2010; Kostons & van der Werf, 2015) and the productive use of intermediate knowledge in the second phase (i.e., by explicitly building on the results of the first phase, e.g., Loibl & Rummel, 2014). Notably, the intermediate knowledge acquired during the first learning phase is not necessarily partial or incomplete target knowledge but can also be instrumental for acquiring the target knowledge in the subsequent learning phase. Such instrumental knowledge can, for instance, comprise situational knowledge that serves as a concrete anchor, or strategic skills that free up cognitive resources in the subsequent learning phase.
In addition to knowledge, motivational and affective states are also changing during the course of a composite instructional design. These non-cognitive states likewise influence learning processes and outcomes. For instance, in a study by Belenky and Nokes-Malach (2012) an initial invention activity in comparison to explicit instruction led to higher mastery orientation after the first phase, which then seemed to impact the learning processes during the subsequent worked-example phase as evident by the performance on a knowledge transfer posttest. Within the CID framework these non-cognitive internal states can be modelled in parallel to the knowledge level (i.e., an initial affective or motivational state that evolves towards an intermediate and a final state and influences the learning processes in the corresponding phases), that also interacts with the other levels. As research on composite instructional designs so far rarely considers affective-motivational aspects (e.g., DeCaro et al., 2015; Glogger-Frey et al., 2015; Sinha, 2022) and for the sake of simplicity, in our analyses we focus primarily on cognitive dimensions (see Fig. 1).
We wish to emphasize that like any other framework, the general CID framework, with its elements and relations as displayed in Fig. 1, is of course “insufficiently specified to enable predictions to be derived” (Anderson, 1993, p. 2). However, by specifying the elements within the CID framework for a concrete instructional design with multiple phases, it is possible to derive theoretical explanations and testable predictions.
Applying the CID framework to composite instructional designs
The CID framework serves as a schema with which to structure the theoretical assumptions on learning mechanisms of composite instructional designs, and in doing so, to guide empirical research. To this end, we suggest that research on the learning mechanisms underlying composite instructional designs should include the following two steps: (1) specifying the elements of the framework and (2) analyzing variations in all three levels while manipulating the instructional design.
(1) In the first step, one has to specify the elements of the framework on the levels of knowledge, learning, and instruction. As the three levels are theoretically highly interdependent, there is no canonical order for this analysis. In many cases, it is reasonable to first define the targeted final knowledge (K2). The learning processes and the instructional design may only be understood in close relation to each other (I1→L1, I2→L2). As a central category, intermediate knowledge (K1) requires two perspectives: It can be considered as final knowledge of the first phase (I1→L1→K1) or as prior knowledge of the second phase (K1 + I2→L2). In order to specify the elements of the framework, the following questions must be answered:
-
What are the relevant knowledge components at the different points in time (i.e., prior knowledge K0, intermediate knowledge K1, and final knowledge K2)? A precise analysis of the hypothesized changes in the knowledge components helps to identify the relevant learning processes that can lead to these changes. As different types of knowledge are acquired via different learning processes (Koedinger et al., 2012), the analysis regarding the knowledge components should also take the type of knowledge into account (e.g., conceptual knowledge, procedural knowledge, associative knowledge).
-
What are the learning processes (L1, L2, e.g., activating prior knowledge, sense making, refinement, fluency building, etc.) that build upon prior knowledge in the first phase or upon intermediate knowledge in the second phase and that lead to intermediate knowledge in the first phase and final knowledge in the second phase, respectively? These assumptions regarding the learning processes should be made explicit for each instructional design (i.e., each condition implemented in a study).
-
How is the instructional design of each of the two phases (I1, I2) and their sequence intended to foster the learning processes?
Specific ways of transforming knowledge via learning processes (K0→L1→K1 or K1→L2→K2) are often denoted by the term “learning mechanisms”. It is these learning mechanisms and their facilitation by specific instructional elements (I1, I2) that are the core interest of research guided by the CID framework.
(2) In the second step, in order to test the assumed learning mechanisms, one has to vary the instructional design such that it changes (if possible) one mechanism at a time and analyze whether further learning mechanisms are influenced by this variation. This includes all elements that are causally dependent on this variation, in particular the intermediate knowledge and its impact on the learning processes in the second phase (K1→L2).
Although there has been increasing interest in research on composite instructional designs, many learning mechanisms and the interplay of the two (or more) phases are not yet well understood. We argue that understanding the effects of a composite instructional design depends crucially on understanding the modification of the learning mechanisms in the second phase due to the previously acquired intermediate knowledge. The proposed CID framework allows for a theoretical analysis in this respect.
Exemplary applications of the CID framework
To exemplify the benefits of analyzing composite instructional designs using the CID framework, we apply the framework to three examples from existing research on composite instructional designs. These examples have different degrees of complexity in terms of the instructional elements involved and the research design.
The first study (Rau et al., 2017) investigated the preparatory effect of one knowledge type for acquiring another knowledge type in a two-phase instructional design (here: conceptual knowledge and procedural knowledge). Here, we seek to demonstrate that the inclusion of all three levels of CID in the analysis can shed new light on Rau et al.’s (2017) findings: Differences in the type of intermediate knowledge (K1) and its specific impact on the learning processes (L2) in the second instructional phase (I2) can help explain the strength of the preparatory effect of the first instructional phase (I1).
The second study (Loibl & Rummel, 2014) is an example of research focusing on the preparatory effect of one instructional phase on another (in this case: problem solving followed by direct instruction) compared to the reverse order. Here, we wish to demonstrate that the differing effects yielded by presenting two instructional elements (I1, I2) in different orders can be explained by the different intermediate knowledge (K1) resulting from the first phase (I1) in one order, as compared to the reverse order, and by the different learning processes which therefore occur in the second phase (L2).
Finally, the third example juxtaposes two seemingly similar studies (Glogger-Frey et al., 2015, 2017), each of which compared two different composite instructional designs (in both studies: problem solving vs. solved solutions followed by direct instruction, i.e., I1 + I2 vs. I’1+I2). The analysis with the CID framework helps to explain the contradicting results of these two studies by pointing at differences in the instruction (I1 and I’1) and the consequences regarding the differing quality of intermediate knowledge resulting from this first phase in the two studies, and their effect on the learning processes in the second phase (K1 + I2→ L2).
The overarching point that we aim to establish across all three examples is that the preparatory effect of an instructional element (I1) on the learning processes of a subsequent phase (L2) depends not only on the pedagogy of this instructional element but also on the resulting intermediate knowledge (K1) and its impact on the following learning processes initiated by the instruction in this phase (K1 + I2→ L2). While this statement might seem rather self-evident, recent research on composite instructional designs often fails to reach down to this level of explicating assumptions on learning mechanisms, thus leaving open questions about seemingly inexplicable or conflicting results, as we illustrate in the following sections.
First example: the impact of different types of intermediate knowledge on subsequent learning processes
Rau et al. (2017) compared the preparatory effect of one knowledge type on acquiring another knowledge type in a two-phase instructional design to support students’ acquisition of representational competencies when learning fractions (e.g., translating between different graphical representations such as area model and number line) (I→K). In terms of targeted final knowledge (K), the authors distinguished between two types of knowledge, which they defined as follows: (K) “sense making of connections by verbally explaining how different representations map to one another”, and (K’) “perceptual fluency that allows students to fast and effortlessly use perceptual features to make connections among representations” (p. 331). Both types of knowledge require a distinction between perceptual features, which show conceptually meaningful information (e.g., parts and wholes), and surface features, which are conceptually irrelevant (e.g., color). This distinction involves either understanding of the conceptual information (K) or procedural pattern recognition (K’).
The authors implemented two different types of instructional designs: In the so-called sense-making activities (I), students receive a worked example of a task (e.g., compare two equivalent fractions) with one representation (e.g., area model) and a corresponding problem with another representation (e.g., number line), presented side by side. After solving the corresponding problem, students receive menu-based reflection prompts to foster integration across the graphical representations (I→K). In the so-called fluency-building activities (I’), students receive a mix of graphical representations and are asked to visually sort the representations into sets of equivalent representations using drag and drop.
Due to the obvious correspondence in the instructional designs (I, I’) and the targeted knowledge components (K, K’), the authors assumed that the sense-making activities would foster conceptual knowledge and that the fluency-building activities would foster fluent procedural knowledge. Ultimately, the focus of their study was on the interplay of the two types of activities. Regarding this interplay, the authors presented two hypotheses derived on the basis of different assumed learning mechanisms (K’+I→L→K, K + I’→L’→K’): (1) Fluent procedural knowledge may free up cognitive resources for complex reasoning (Koedinger et al., 2012); therefore, implementing fluency-building activities prior to sense-making activities (I’ before I) may result in fewer errors in the sense-making activities and support the acquisition of conceptual knowledge (fluency-first hypothesis). (2) As fluency-building activities are based on the premise that students already know which perceptual features are meaningful, implementing sense-making activities prior to fluency-building activities (I before I’) may result in fewer errors in the fluency-building activities and support the acquisition of procedural knowledge (sense-making-first hypothesis). The authors formulated no theoretical assumptions regarding different strengths of the two hypotheses.
In their study, the authors compared the two sequences (sense-making first or fluency-building first) in order to find support for one or both of the hypotheses. For this purpose, they compared (a) the error rates in one type of instructional activities with or without preceding activities of the other type and (b) both types of knowledge tested after the combined instructional activities. Figure 2 summarizes the specifications of the CID framework for the presented study as a basis for predicting (or explaining) the results of the study, with a focus on the effect of the intermediate knowledge on the learning processes in the second phase (K1→L2).
Application of the CID framework to the sense-making first and fluency-building first conditions of Rau et al. (2017)
When procedural knowledge (K1 in the upper part of Fig. 2) has been acquired, the learner can make use of his or her ability to effortlessly make connections among representations based on perceptual features when integrating different representations. This effortless process frees up cognitive capacity (Koedinger et al., 2012) when reasoning about the different representations (L2). In turn, these reasoning processes foster conceptual knowledge on mapping different representations via meaningful features. This mechanism is rather general: Intermediate knowledge frees up cognitive resources for the subsequent phase. Moreover, it should ease learning in the second phase whenever the acquired knowledge is applicable - at least if the second phase is cognitively demanding. Nevertheless, it does not alter the learning processes of the second phase (L2) in principle, and as the learning processes of the second phase are not altered, it seems reasonable to expect rather small effects regarding the fluency-first hypothesis.
When conceptual knowledge on the meaningful features of different representations (K1 in lower part of Fig. 2) has been acquired, subsequent pattern recognition is facilitated (L2) by building on this knowledge, because this process requires the learner to distinguish between meaningful and irrelevant perceptual features. Pattern recognition processes foster the acquisition of procedural knowledge, that is, the effortless connection among representations based on perceptual features. Without this knowledge (i.e., when the visual sorting task comes first), the process of pattern recognition may include not only relevant features but also irrelevant surface features. As a consequence, learners may not learn to effortlessly make meaningful connections among representations. In this sequence, the intermediate knowledge (K1) thus alters the learning processes (L2) of the second phase. Therefore, this preparation mechanism in the sense-making-first sequence likely substantially fosters learning in the second phase of this condition. Indeed, the study supports the sense-making-first hypothesis, but not the fluency-first hypothesis, regarding the error rates in the activities as well as the knowledge measured after the combined instructional activities. Similar to our analysis, in the discussion of their findings, Rau et al. (2017) suggest that students may need conceptual knowledge to identify conceptually relevant perceptual features for efficient and correct mapping of representations in the fluency-building activity. However, they do not compare the different types of intermediate knowledge acquired in the two conditions to explain the different strengths of the preparatory mechanisms.
By including the knowledge level in our analysis, we identified different levels of specificity of the preparation mechanism in the two investigated conditions - a very general mechanism of cognitive easing (Koedinger et al., 2012) in the fluency-first condition and the acquisition of very specific knowledge components as unique prerequisites for the second phase in the sense-making-first condition. Of course, the assumption that the specificity of the preparation mechanism is important for explaining the effects of composite instructional designs would need to be tested with a new dataset.
Second example: altered learning processes by switching instructional phases
Research on problem-solving prior to instruction (PS-I) often investigates the effect of PS-I in comparison to the opposite sequence of the two phases, i.e., I-PS (Loibl et al., 2017; Sinha & Kapur, 2021). With regard to the targeted final knowledge (K2), the majority of the studies are conducted in the area of statistics, with the target concept of variance, and differentiate between procedural knowledge (i.e., applying the learned formula for standard deviation on a new set of data) and conceptual knowledge (i.e., reasoning on the components of the formula, such as using absolute or squared deviations to prevent positive and negative deviations from cancelling each other out).
With regard to the instructional design, in the problem-solving phase (I1 or I2 depending on the order), learners are given multiple datasets (e.g., the number of goals scored by three soccer players in each of the last 10 or 20 years, Kapur, 2012; Loibl & Rummel, 2014) and are asked to find the most consistent dataset (i.e., the most consistently scoring soccer player). In the explicit instruction phase (I2 or I1), the formula for variance is introduced by explaining the reasoning behind each of the components of the formula and providing an example of how to apply the formula to a given dataset. The explicit instruction phase may or may not also include the discussion of suboptimal solutions (Loibl & Rummel, 2014).
With regard to the learning processes in PS-I, Loibl et al. (2017) discuss: (i) activating prior knowledge during the initial problem-solving phase (L1), (ii) becoming aware of knowledge gaps (L1 and/or L2, depending on the exact instructional design), and (iii) recognizing relevant features of the target concept during subsequent instruction (L2). Assumptions regarding the learning processes in the opposite order of phases, i.e., I-PS, are not made explicit.
The discussed learning processes are clearly linked to the knowledge level (L→K): Activating prior knowledge results in activated knowledge (K1); becoming aware of knowledge gaps culminates in an awareness of knowledge gaps and possibly negative knowledge regarding how not to solve the task (K1); recognizing relevant features and integrating them into the prior knowledge structure leads to the final knowledge (K2). However, links between the instructional design and the learning processes (I→L) remain speculative because the effects of different instructional designs on single learning processes are not investigated and learning processes usually are not assessed systematically.
In summary, to date, the literature on PS-I (a) does not link all three levels when discussing the mechanisms and (b) does not specify the mechanisms of the comparison condition I-PS. Applying the CID framework may therefore inform research on PS-I with regard to the interplay of the three levels and the altering of mechanisms when changing the design or the sequence of the two phases (Loibl et al., 2023).
Most studies on PS-I compare variants of PS-I to I-PS and demonstrate beneficial effects of PS-I on conceptual knowledge outcomes (e.g., DeCaro & Rittle-Johnson, 2012; Kapur & Bielaczyc, 2012; Loibl & Rummel, 2014; Weaver et al., 2018, for overviews see Loibl et al., 2017; Sinha & Kapur, 2021) and transfer of knowledge (e.g., Belenky & Nokes-Malach, 2012; Jacobson et al., 2017; Lai et al., 2017; Schwartz et al., 2011). We apply the CID framework to explain this outcome. The instructional variants in this research are represented in Fig. 3 and will be discussed in the following. Based on the study by Loibl and Rummel (2014), we distinguish between:
-
➀ PS-I with building on typical student solutions
-
➁ PS-I without building on typical student solutions (i.e., standard instruction)
-
➂ I-PS with building on typical student solutions
-
➃ I-PS without building on typical student solutions (i.e., standard instruction)
Early research on PS-I compared ➀ PS-I with building on student solutions to ➃ I-PS with standard instruction and found beneficial effects of PS-I on conceptual knowledge but not on procedural knowledge (Loibl et al., 2017). This is unsurprising given that both the different sequence of the phases and the different instructional design of the explicit instruction phase (explanation with or without building on student solutions) lead to learning processes that either foster mostly conceptual knowledge (➀ PS-I with student solutions) or mostly procedural knowledge (➃ I-PS with standard instruction). Due to the confound between sequence and instructional design of the phases, the learning effects cannot be attributed to one of these instructional elements.
Application of the CID framework to the PS-I and I-PS conditions in Loibl and Rummel (2014)
Loibl and Rummel (2014) addressed this confound by implementing all four conditions. They found a significant interaction insofar as the PS-I sequence was only superior to I-PS in terms of acquiring conceptual knowledge (K2) when the instruction (I2) was building on student solutions. More precisely, for conceptual knowledge at posttest (K2), PS-I with student solutions (➀) outperformed I-PS with student solutions (➂), which in turn outperformed PS-I and I-PS without student solutions (➁, ➃), and there were no significant differences between the latter two conditions.
On the learning process level, Loibl and Rummel (2014) discussed learning processes that may take place in PS-I. In the initial problem-solving phase, students activate their prior knowledge and by struggling with the problem at hand, they become aware of their knowledge gaps (called global awareness of knowledge gaps) (L1). By illustrating the limitations of the typical erroneous student solutions, the subsequent explicit instruction phase creates an awareness of specific gaps. As the presented components of the concept of variance resolve these specific gaps, students likely focus on processing these components (L2). While the assumed learning processes in the PS-I condition with student solutions (➀) are thus defined for both phases, this definition is missing for the other conditions.
On the instruction level, I-PS (with building on student solutions) seems to constitute an adequate comparison condition to test the effect of sequence (➂). However, when explicitly considering the learning process level, it becomes apparent that changing the instructional sequence (PS-I vs. I-PS) has multiple effects: Delaying the explicit instruction phase not only allows negative knowledge and encoded examples from the preceding problem-solving phase (intermediate knowledge, K1) to be linked to the processed explanation, but also completely changes the processes in the problem-solving phase: In PS-I, the learning processes in the problem-solving phase are activating prior knowledge and identifying knowledge gaps (L1). Together, these processes foster negative knowledge and encoded examples as early forms of conceptual knowledge (L1→K1). In I-PS, the learning processes in the problem-solving phase, which comes second here, are practicing the application of the learned formula, which does not impact conceptual knowledge but rather only strengthens procedural knowledge (L2→K2).
To summarize, by including all three levels of the CID framework in our analysis, we demonstrated that changing the instructional design (in this case the order of the instructional phases) affects multiple mechanisms on the other two levels. Accordingly, investigating single mechanisms requires research strategies that not only carefully control the design on the instruction level but also take into account resulting changes on the learning process level and the knowledge level in all implemented conditions.
Third example: explaining inconsistent results: the impact of different qualities of intermediate knowledge to prepare learning from subsequent instruction
Schwartz et al. (2011) as well as Glogger-Frey et al. (2015, 2017) conducted studies investigating students’ conceptual understanding of ratios in physics. Similar to the studies described in the second example above, Schwartz et al. compared the learning effect of PS-I to that of I-PS (I→K) in two experiments: In I-PS, students received explicit instruction on the relevant concepts and formulas (e.g., density) (I1) before being provided with a practice problem (I2). In PS-I, students were asked to invent formulas based on the problem-solving material provided (I1) and received explicit instruction on the relevant concepts and formulas only afterwards (I2). When comparing the learning effect of these instructional designs (I→K), the authors found beneficial effects for PS-I. However, as our analysis of PS-I research in the previous sections revealed, a comparison between PS-I and I-PS does not allow for a systematic investigation of the underlying learning mechanisms due to the multiple differences occurring on the learning process and knowledge levels. An alternative research strategy is to change the instructional designs of one phase while keeping the order of the phases identical.
Glogger-Frey and colleagues (Glogger-Frey et al., 2015; Glogger-Frey et al., 2017) investigated different designs of the first instructional phase (I1): Students were either asked to invent an index for density (I1) before receiving explicit instruction on the scientific formula for density and other ratios in physics (I2) in a PS-I condition, or were given a worked-out solution of the invention problem (I’1) before explicit instruction (I2) in a WS-I condition (worked solution prior to instruction).
The authors conducted two studies using this design. The sole difference between the two studies lay in the duration of the initial phase (I1, I’1). Students in the first study (Glogger-Frey et al., 2015) went through one problem (short PS1/WS1: find a crowdedness index of clowns in a bus), whereas students in the second study (Glogger-Frey et al., 2017) went through two problems during the first learning phase (long PS2/WS2: find a crowdedness index, find an index for quality of gold).
The results of the first study (Glogger-Frey et al., 2015; short PS1/WS1) revealed that PS-I was less beneficial for transfer than explaining the worked solution before instruction (WS-I; I→K). The second study, with the long problem-solving or worked-solution phase (Glogger-Frey et al., 2017), revealed that PS-I was more beneficial for transfer than explaining the worked solution before instruction (WS-I). Long problem-solving phases (i.e., the length of multiple lessons) were also used in Kapur’s studies (e.g., Kapur, 2012) and in Schwartz et al. (2011), while shorter phases (i.e., one lesson) were employed in the studies by Loibl and Rummel (e.g., 2014). To date, no theoretical arguments have been put forward as to why a shorter versus longer PS phase would alter the PS-I effects if other characteristics of the intervention remain the same (Sinha & Kapur, 2021). Therefore, we use the CID framework to explain the contradictory findings of the two studies (Glogger-Frey et al., 2015 vs. 2017).
The targeted final knowledge (K2) comprises conceptual and procedural knowledge on the concept of density, and on a higher-order level, the concept of ratios in physics. The target conceptual knowledge comprises knowing that two quantities can be commensurable by creating a ratio and thus a new quantity. Transfer of conceptual knowledge was targeted, that is, the ratio concept had to be applied in a context different from the density and other instructed contexts (e.g., determining the stiffness of trampoline fabrics, which is the equivalent to finding their spring constants). The target procedural knowledge consists of knowing how to apply the script of calculating density or other specified ratios from given numbers.
The variants of the instructional design in these studies are:
-
➀ short PS1-I: one problem-solving task prior to instruction
-
➁ short WS1-I: one worked solution prior to instruction
-
➂ long PS2-I: two problem-solving tasks prior to instruction
-
➃ long WS2-I: two worked solutions prior to instruction
In order to carve out the central differences in the CID analysis, we focus our discussion on the first three variants (see Fig. 4).
In the first phase of the PS1-I condition (Fig. 4, part ➀), students analyze material depicting clowns in buses from different companies to find each company’s crowdedness index (I1). To do this, they compare and contrast the companies (L1). While encoding the examples of differently crowded buses, students notice (L1) potentially relevant (i.e., number of clowns, number of bus compartments) or irrelevant features (surface features, e.g., clown clothing, exact positions). Students attempt to find solutions to combine these features, potentially including irrelevant features. This process leads to encoded examples, including relevant and irrelevant features (K1).
In the first phase of the WS1-I condition (Fig. 4, part ➁), students receive the same material as in the PS1-I condition, but with an additional worked solution showing the correct solution and a modeled reasoning process. Students are asked to explain the solution using this material (I1). To solve this task, students encode the worked solution (L1). However, unlike the PS1-I condition, WS1-I students likely focus on relevant features and ignore irrelevant surface features when comparing the companies because only relevant features are addressed in the worked solution. This process leads to encoded examples, including relevant features only (K1).
Regarding the learning processes (L1), the most important difference is that the PS1-I condition encodes both irrelevant and relevant features and potentially uses them in suboptimal solution ideas. In contrast, WS1-I focuses only on relevant features, resulting in intermediate knowledge (K1) with fewer surface features than in PS1-I. These assumptions are supported by data in the short-preparation study (Glogger-Frey et al., 2015): The intermediate knowledge, measured by a recall test, encompassed fewer surface features in the WS1-I than in the PS1-I condition.
In the second phase of both PS1-I and WS1-I, students are instructed about the scientific formula for density and other ratios in physics (I2). During this phase, they connect the encoded examples, including relevant features, to the instructed formulas (L2). PS1-I students additionally need to sort irrelevant surface features from relevant features (L2), while WS1-I students do not. Thus, WS1-I students can potentially build knowledge more efficiently. The final knowledge (K2) differs accordingly: PS1-I students may retain some surface details of the material and have less structured knowledge than WS1-I students, resulting in different posttest performance.
In contrast to these findings, in the long-preparation study (Glogger-Frey et al., 2017), students in the PS2-I condition outperformed students in the WS2-I condition. As opposed to instructional design variants ➀ and ➁, students in the PS2-I condition (Fig. 4, part ➂) work on problems embedded in two different contexts (I1). By contrasting examples in two contexts, they have the opportunity to notice the irrelevance of surface features (L1). Moreover, they also have the opportunity to notice more relevant features and can more often combine features through the second problem with the same structure. In the words of the literature on analogical reasoning, providing two problems instead of just one can induce analogical encoding. That is, learners map similarities in the deep structure of the two problems instead of focusing on surface features (Gentner et al., 2003). Indeed, the solution rate in problem 1 and problem 2 in Glogger-Frey et al. (2017) rose from 34 to 56%. As a consequence, intermediate knowledge after the second problem (K1) should comprise fewer surface features and instead negative knowledge in terms of explicit knowledge that specific surface features are irrelevant. In addition, the intermediate knowledge should be more generalized than after working with just one problem through analogical encoding (Renkl, 2014; Gentner, 1983, 1989; Gentner et al., 2003). In consequence, less sorting of surface from deep features (L2) is necessary when processing instruction (I2) compared to a PS1-I condition with just one problem. The longer WS2-I condition (Fig. 4, part ➃), meanwhile, cannot significantly reduce encoded irrelevant features (in comparison to WS1-I), since few of them were encoded in the first problem. Without having encoded irrelevant features, students in this condition (in comparison to PS2-I) are less likely to acquire explicit negative knowledge regarding the irrelevance of surface features.
In summary, the detailed analysis using the CID framework helps to explain the seemingly inconsistent results reported by Glogger-Frey and colleagues (2015 vs. 2017). The key to this explanation lies in the differences in the quality of intermediate knowledge. We argue that the intermediate knowledge must be sufficiently comprehensive and structured to provide fertile ground for learning from subsequent instruction. However, the intermediate knowledge can be fostered by different types of instructional designs (“different roads lead to Rome”, as Renkl, 2015, put it) as long as the instructional design triggers learning processes, leading to an adequate quality of intermediate knowledge in a composite instructional design. Besides the type of design, the characteristics of the learner also affect intermediate knowledge: More practiced or gifted learners can arrive at high-quality intermediate knowledge with less support (Lim et al., 2023). While future studies with a shared sample need to investigate whether the comparison across studies made here is valid, our line of argument is supported by the solution rates (indicating well-structured intermediate knowledge) and effects reported in different studies: Studies with low solution rates (28% in Glogger-Frey et al., 2015, 14% in Roelle & Berthold, 2016) in the initial problem-solving phase do not find beneficial effects of PS-I; studies with high solution rates (80% in Schwartz et al., 2011, 90% in Schalk et al., 2018) in the initial problem-solving phase do find beneficial effects of PS-I.
Discussion
Insights gained from re-analyzing studies on composite instructional designs using the CID framework
Instructional designs that combine multiple phases with different pedagogies and goals are increasingly researched, but studies investigating such composite instructional designs have frequently yielded inconclusive findings (e.g., Glogger-Frey et al., 2015 vs. 2017; Hartmann et al., 2022 vs. Kapur, 2014). Using the CID framework, it is possible to analyze and design research on composite instructional designs and to explain the existing findings by connecting the knowledge, learning process, and instructional design levels (Koedinger et al., 2012) and linking these levels across phases. The pivotal element of the analysis is the identification of intermediate knowledge components after the first phase and their interaction with learning mechanisms of the second phase.
By analyzing exemplary research using our CID framework, we were able to uncover some important insights that were not hitherto apparent in previous interpretations of the studies: The first study (Rau et al., 2017) represented an example of combining instructional designs that target different types of knowledge. The analysis with the CID framework showed that differences in the strength of the preparatory effect of one instructional phase on learning in the subsequent phase can be explained by the type of intermediate knowledge and its specific effect on subsequent learning processes. The acquired intermediate knowledge components might be specific prerequisites for subsequent learning processes or may free up cognitive capacity for subsequent learning processes in an unspecified manner, that is, without being necessary and sufficient prerequisites.
The second study (Loibl & Rummel, 2014) was an example of research investigating the preparatory effect of one type of instructional phase on another in comparison to the reverse order. The analysis with the CID framework showed that by changing the order of the instructional phases, more than one learning mechanism is altered, as the intermediate knowledge impacts learning processes in the second learning phase in both implemented orders. Thus, the CID framework highlighted a limitation of research on composite instructional designs that merely considers the instruction level and often only discusses learning mechanisms of one order, but not the other.
Finally, the third example juxtaposed two seemingly similar studies (Glogger-Frey et al., 2015, 2017) that compared the preparatory effect of two different instructional designs. The analysis with the CID framework helped to explain the inconsistent results reported in these two studies by pointing to different qualities of intermediate knowledge in these two studies and the effect on the learning processes in the second phase.
Overall, these analyses highlighted the need to consider all three levels of the CID framework in order to understand the effects of composite instructional designs. The exemplary analysis of previous research demonstrated the power of the CID framework for understanding surprising or conflicting results and for building consistent theory. The CID framework can also support the design of new studies that are suitable for investigating the effect of composite instructional designs.
In general, different types of intermediate knowledge (K1) can be activated or generated (L1) in the first instructional phase (I1), and these different types of intermediate knowledge (K1) can prove beneficial in the second phase. The first phase, for instance, contributes to activating or generating (L1) (1) partial or erroneous knowledge (concepts, procedures), which serves as a component for integration processes (L2) during the second phase. This can be, for example:
(1a) partial knowledge, which is integrated into target knowledge,
(1b) erroneous knowledge, which (via refutation) contributes to target knowledge,
(1c) specific negative knowledge, which supports the integration by focusing on relevant aspects of target knowledge.
There are also types of intermediate knowledge that act as (2) instrumental knowledge. It is assumed that these knowledge types do not become part of the target knowledge but rather support or mediate its construction, for example:
(2a) situational knowledge, which serves as a concrete anchor for the construction of (more abstract) target knowledge,
(2b) unspecific negative knowledge (i.e., global awareness of knowledge gaps), which generates motivation to overcome knowledge gaps,
(2c) supportive knowledge (e.g., strategies, fluency), which frees cognitive capacity for the construction of target knowledge.
As illustrated in Fig. 5, these different types of intermediate knowledge (K1) contribute differently to the learning processes (L2) initiated by the instructional design of the second phase (I2). To achieve a better understanding of the mechanisms of composite instructional designs, it seems reasonable to manipulate and/or measure these different types of intermediate knowledge separately, wherever possible.
Methodological challenges when applying the CID framework to design future research on composite instructional designs
Research on composite instructional designs should specify the elements of the CID framework at all three levels and in all phases for each implemented condition, with a specific focus on the intermediate knowledge and its impact on the learning processes of the second phase. To advance research on composite instructional designs, we need to formulate and test causal assumptions on learning mechanisms. This requires experimental settings in which the instructional design is systematically manipulated in such a way that it only alters single or a small number of learning processes, in a controlled way. Furthermore, we should strive to acquire, and include in the evaluation, any data that reflect relevant aspects of the learning processes and the knowledge components.
As discussed above, changing the order of two instructional phases alters the learning processes in both instructional phases, making it impossible to derive specific theoretical and empirical conclusions. It therefore seems more promising to change the instructional design within one phase without altering the order of the phases. In which regard the instructional design in the variants should differ depends on the specific research question. For instance, one could manipulate which knowledge components are activated in the first instructional phase by implementing variants of vicarious problem solving (i.e., examples of erroneous or incomplete problem-solving attempts, Brand et al., 2021; Hartmann et al., 2022). Vicarious problem solving (in contrast to one’s own problem solving) enables a systematic manipulation of which knowledge components are addressed in the solution attempts. Alternatively, one could foster negative knowledge in the first phase by presenting specific contrasting examples or specific feedback. These manipulations should lead to different intermediate knowledge which, in turn, impacts learning in the second phase.
Regarding the learning mechanisms of the second instructional phase, one could investigate the extent to which linking the intermediate knowledge to the explanation is relevant for learning. This question has already been partially addressed by Loibl and Rummel (2014) in their comparison of PS-I with and without building on student solutions. PS-I was only effective when the explanation built on typical student solutions. However, the study failed to test the connection between intermediate knowledge and knowledge integration directly: Neither intermediate knowledge nor indicators of the learning processes in the second phase were assessed. Therefore, it was not possible to investigate the connection between the individual intermediate knowledge and the learning processes triggered by the explanation that built on typical student solutions.
To systematically manipulate the assumed learning processes through instructional elements, one needs precise theoretical assumptions on the connection between the instructional elements and processes. For example, learning processes can be increased or reduced by prompts (e.g., elaboration prompts or metacognitive prompts, Nückles et al., 2020; Berthold et al., 2007) or by the salience of certain information to be processed during learning (e.g., by signaling, Richter et al., 2016, or well-designed contrasting cases, Chin et al., 2016).
Beyond systematically manipulating learning processes, one should also attempt to measure these. Traditional methods, such as think-aloud approaches, can procure valuable insights into student thinking without substantially interfering the learning process (Fox et al., 2011). However, it is effortful to stimulate verbalizations and learners may fall silent when their thinking would get interesting for the researcher. Furthermore, recording think-aloud data may not be possible in full classrooms and the necessity to transcribe and code the data allows only small sample sizes. In this situation computer-based settings can create good opportunities to gather data on the learning process, also without interfering with the learning processes. Additionally, they may procure data also for large populations and avoid high coding expenses. However, interpreting such data requires theoretical assumptions on how the (internal) learning processes and the observable activities correlate (Goldhammer et al., 2021; Reinhold et al., 2024). Fortunately, there is a broad variety of approaches for how to establish such a link between behavior and cognition on a theoretical and empirical level (Goldhammer et al., 2021; Huber & Bannert, 2023; Mislevy et al., 2012). A smart combination of think-aloud analyses in explorative research phases can aid to generate valid theoretical assumptions which then can be further explored and tested with computer-based process data with larger sample sizes.
As a key element of composite instructional designs, the measurement of intermediate knowledge can deliver valuable empirical information. However, intermediate knowledge states between two instructional phases, such as not yet fully developed propositional networks or non-verbal implicit knowledge, can be fragile and hard to measure. Furthermore, any measurement may influence knowledge states in a way that interacts with the assumptions on the effects of the instruction in the preceding or subsequent phase.
Because we assume that intermediate knowledge explains the extent to which learners learn from the subsequent instructional phase, mediation analyses can be a highly valuable method to substantiate such assumptions, as they can illuminate how variables causally affect each other (Kenny, 2023). An “experiment that includes an assessment of potential mediators can significantly illuminate the mechanism by which the intervention has an impact upon the [dependent variable] DV” (Jose, 2016, p. 335). While some studies have already included mediation analyses, future research needs to include larger sample sizes and investigate more of the theorized learning mechanisms.
To be able to pinpoint single mechanisms, studies need to be highly controlled. For instance, implementing complex problem solving or inquiry learning in the first phase does not allow for controlling the exact learning processes, and is therefore not suitable for identifying the effect of negative knowledge on subsequent learning processes. Instead, instructional designs with a high level of control of the learning processes leading to negative knowledge are needed. Nevertheless, such controlled studies do not account for the complexity in full composite instructional designs that include problem solving or inquiry. In order to draw consequences for complex composite instructional designs from such controlled studies, one needs to acknowledge in how far the observed phenomena are restricted due to the controlled setting. The optimal amount of control thus depends on the research question under consideration. For example, if one is interested in the general effect of inquiry learning on (intermediate) negative knowledge, a high control of this phase would not fit to the research question. Also, in some cases, it may make sense to conduct more controlled studies regarding single mechanisms only after establishing an overall effect of the instructional design in question.
Conclusion
Our analyses of composite instructional designs using the CID framework suggest that intermediate knowledge plays a key role in explaining the mechanisms at play, as it alters subsequent learning processes. Research on composite instructional designs can be advanced by formulating and testing causal assumptions on these learning mechanisms.
Moreover, the CID framework has relevant practical significance. Beyond being a tool for research, it can also serve as a thinking tool in practical situations of designing everyday classroom teaching or instructional material such as textbooks. Having a clear picture of what students are supposed to learn, of what their intermediate ideas after an advance-organizing phase or a discovery phase are, and of how to build on these in a subsequent phase, are prerequisites for developing well-designed learning environments (Prediger et al., 2021).
Questions regarding the effectiveness of different pedagogical approaches, such as inquiry learning and instructional explanations, have recently become a hot topic once again. In this context, de Jong et al. (2023) make a plea for researchers to combine such approaches, rather than merely comparing them. The paradigm of composite instructional designs that we discuss in this article can be an invaluable remedy on the path to overcoming the counterproductive quarrels between proponents of different single-phase instructional designs with only one pedagogical approach.
The proposed CID framework may provide an analytical lens and heuristic tool, for researchers and practitioners alike, to more systematically conceive composite instructional designs that help to truly understand and utilize the learning mechanisms at play. This can be considered as a starting point to develop a fully grown paradigm of composite instructional designs.
References
Anderson, J. R. (1993). Rules of the mind. Lawrence Erlbaum.
Anderson, J. R., & Lebiere, C. (1998). The atomic components of thought. Lawrence Erlbaum Associates.
Ausubel, D. P. (1960). The use of advance organizers in the learning and retention of meaningful verbal material. Journal of Educational Psychology, 52, 267–272. https://doi.org/10.1037/h0046669.
Barnett, S. M., & Ceci, S. J. (2002). When and where do we apply what we learn? A taxonomy for far transfer. Psychological Bulletin, 128(4), 612–637.
Belenky, D. M., & Nokes-Malach, T. J. (2012). Motivation and transfer: The role of mastery-approach goals in preparation for future learning. Journal of the Learning Sciences, 21(3), 399–432.
Berthold, K., Nückles, M., & Renkl, A. (2007). Do learning protocols support learning strategies and outcomes? The role of cognitive and metacognitive prompts. Learning and Instruction, 17(5), 564–577.
Brand, C., Hartmann, C., Loibl, K., & Rummel, N. (2021). Observing or Generating Solution Attempts in Problem Solving Prior to Instruction: Are the Preparatory Processes Comparable? In Proceedings of the 15th International Conference of the Learning Sciences-ICLS 2021 International Society of the Learning Sciences.
Chin, D. B., Chi, M., & Schwartz, D. L. (2016). A comparison of two methods of active learning in physics: Inventing a general solution versus compare and contrast. Instructional Science, 44, 177–195. https://doi.org/10.1007/s11251-016-9374-0.
Cronbach, L. J., & Snow, R. E. (1977). Aptitudes and instructional methods: A handbook for research on aptitude-treatment interactions. Irvington.
de Jong, T., Lazonder, A. W., Chinn, C. A., Fischer, F., Gobert, J., Hmelo-Silver, C. E., Koedinger, K. R., Krajcik, J. S., Kyza, E. A., Linn, M. C., Pedaste, M., Scheiter, K., & Zacharia, Z. C. (2023). Let’s talk evidence–the case for combining inquiry-based and direct instruction. Educational Research Review, 39, 100536. https://doi.org/10.1016/j.edurev.2023.100536.
DeCaro, M. S., & Rittle-Johnson, B. (2012). Exploring mathematics problems prepares children to learn from instruction. Journal of Experimental Child Psychology, 113(4), 552–568. https://doi.org/10.1016/j.jecp.2012.06.009.
DeCaro, D. A., DeCaro, M. S., & Rittle-Johnson, B. (2015). Achievement motivation and knowledge development during exploratory learning. Learning and Individual Differences, 37, 13–26.
Fox, M. C., Ericsson, K. A., & Best, R. (2011). Do procedures for verbal reporting of thinking have to be reactive? A meta-analysis and recommendations for best reporting methods. Psychological Bulletin, 137(2), 316.
Gentner, D. (1983). Structure-mapping: A theoretical framework for analogy. Cognitive Science, 7, 155–170.
Gentner, D. (1989). The mechanisms of analogical learning. In S. Vosniadou, & A. Ortony (Eds.), Similarity and analogical reasoning (pp. 199–241). Cambridge University Press.).
Gentner, D., & Stevens, A. L. (1983). Mental models. Lawrence Erlbaum.
Gentner, D., Loewenstein, J., & Thompson, L. (2003). Learning and transfer: A general role for analogical encoding. Journal of Educational Psychology, 95(2), 393–408. https://doi.org/10.1037/0022-0663.95.2.393.
Glogger-Frey, I., Fleischer, C., Grüny, L., Kappich, J., & Renkl, A. (2015). Inventing a solution and studying a worked solution prepare differently for learning from direct instruction. Learning and Instruction, 39, 72–87.
Glogger-Frey, I., Gaus, K., & Renkl, A. (2017). Learning from direct instruction: Best prepared by several self-regulated or guided invention activities? Learning and Instruction, 51, 26–35. https://doi.org/10.1016/j.learninstruc.2016.11.002.
Glogger-Frey, I., Treier, A. K., & Renkl, A. (2022). How preparation-for-learning with a worked versus an open inventing problem affect subsequent learning processes in pre-service teachers. Instructional Science, 1–23. https://doi.org/10.1007/s11251-022-09577-6.
Goldhammer, F., Hahnel, C., Kroehne, U., & Zehner, F. (2021). From byproduct to design factor: On validating the interpretation of process indicators based on log data. Large- Scale Assessments in Education, 9(1), 20. https://doi.org/10.1186/s40536-021-00113-5.
Gurlitt, J., & Renkl, A. (2010). Prior knowledge activation: How different concept mapping tasks lead to substantial differences in cognitive processes, learning outcomes, and perceived self-efficacy. Instructional Science, 38, 417–433. https://doi.org/10.1007/s11251-008-9090-5.
Gurlitt, J., Dummel, S., Schuster, S., & Nueckles, M. (2012). Differently structured advance organizers lead to different initial schemata and learning outcomes. Instructional Science, 40, 351–369. https://doi.org/10.1007/s11251-011-9180-7.
Hartmann, C., van Gog, T., & Rummel, N. (2022). Productive versus vicarious failure: Do students need to fail themselves in order to learn? Applied Cognitive Psychology, 36(6), 1219–1233.
Huber, K., & Bannert, M. (2023). Investigating learning processes through analysis of navigation behavior using log files. Journal of Computing in Higher Education. https://doi.org/10.1007/s12528-023-09372-3.
Jacobson, M. J., Markauskaite, L., Portolese, A., Kapur, M., Lai, P. K., & Roberts, G. (2017). Designs for learning about Climate Change as a Complex System. Learning and Instruction, 52, 1–14. https://doi.org/10.1016/j.learninstruc.2017.03.007.
Jose, P. E. (2016). The merits of using longitudinal mediation. Educational Psychologist, 51(3–4), 331–341.
Kalyuga, S. (2007). Expertise reversal effect and its implications for learner-tailored instruction. Educational Psychology Review, 19, 509–539.
Kalyuga, S., & Singh, A. M. (2016). Rethinking the boundaries of cognitive load theory in Complex Learning. Educational Psychology Review, 28, 831–852. https://doi.org/10.1007/s10648-015-9352-0.
Kapur, M. (2012). Productive failure in learning the concept of variance. Instructional Science, 40(4), 651–672. https://doi.org/10.1007/s11251-012-9209-6.
Kapur, M. (2014). Comparing learning from productive failure and vicarious failure. Journal of the Learning Sciences, 23(4), 651–677.
Kapur, M., & Bielaczyc, K. (2012). Designing for productive failure. Journal of the Learning Sciences, 21(1), 45–83.
Kenny, D. A. (2023, July 18). Mediation. Retrieved from http://davidakenny.net/cm/mediate.htm.
Kintsch, W., & Greeno, J. G. (1985). Understanding and solving word arithmetic problems. Psychological Review, 92(1), 109.
Koedinger, K. R., Corbett, A. T., & Perfetti, C. (2012). The knowledge-learning‐instruction framework: Bridging the science‐practice chasm to enhance robust student learning. Cognitive Science, 36(5), 757–798.
Kostons, D., & van der Werf, G. (2015). The effects of activating prior topic and metacognitive knowledge on text comprehension scores. British Journal of Educational Psychology, 85(3), 264–275.
Kunter, M. (2005). Multiple ziele Im Mathematikunterricht [Multiple goals in the mathematics classroom]. Waxmann.
Lai, P. K., Portolese, A., & Jacobson, M. J. (2017). Does sequence matter? Productive failure and designing online authentic learning for process engineering. British Journal of Educational Technology, 48(6), 1217–1227. https://doi.org/10.1111/bjet.12492.
Lim, S. A., Jung, J. Y., & Kalyuga, S. (2023). Effectiveness of invention tasks and explicit instruction in preparing intellectually gifted adolescents for learning. Instructional Science. https://doi.org/10.1007/s11251-023-09616-w.
Loibl., K., Leuders, T., Glogger-Frey, I., & Rummel, N. (2023). Cognitive Analysis of Composite Instructional Designs: New Directions for Research on Problem-Solving Prior to Instruction. In C. Damșa, M. Borge, E. Koh, & M. Worsley (Eds.), Proceedings of the 16th International Conference on Computer-Supported Collaborative Learning - CSCL 2023 (pp. 321–324). Montreal, Canada: International Society of the Learning Sciences.
Loibl, K., & Leuders, T. (2018). Errors during exploration and consolidation – the effectiveness of productive failure as sequentially guided discovery learning. Journal für Mathematik-Didaktik, 39(1), 69–96.
Loibl, K., & Rummel, N. (2014). Knowing what you don’t know makes failure productive. Learning and Instruction, 34, 74–85.
Loibl, K., Roll, I., & Rummel, N. (2017). Towards a theory of when and how problem solving followed by instruction supports learning. Educational Psychology Review, 29(4), 693–715.
Mislevy, R. J., Behrens, J. T., Dicerbo, K. E., & Levy, R. (2012). Design and discovery in educational assessment: Evidence-centered design, psychometrics, and educational data mining. Journal of Educational Data Mining, 4(1), 11–48. https://doi.org/10.5281/zenodo.3554641.
Newman, P. M., & DeCaro, M. S. (2019). Learning by exploring: How much guidance is optimal? Learning and Instruction, 62, 49–63.
Nückles, M., Winter, A., Wittwer, J., Herbert, M., & Hübner, S. (2006). How do experts adapt their explanations to a layperson’s knowledge in Asynchronous Communication? An experimental study (Vol. 16, pp. 87–127). User Modeling and User-Adapted Interaction.
Nückles, M., Roelle, J., Glogger-Frey, I., Waldeyer, J., & Renkl, A. (2020). The self-regulation-view in writing-to-learn: Using journal writing to optimize cognitive load in self-regulated learning. Educational Psychology Review, 32, 1089–1126. https://doi.org/10.1007/s10648-020-09541-1.
Oser, F., Hascher, T., & Spychiger, M. (1999). Lernen aus Fehlern—Zur Psychologie des „negativen Wissens [Learning from mistakes—on the psychology of negative knowledge]. In Fehlerwelten: Vom Fehlermachen und Lernen aus Fehlern. Beiträge und Nachträge zu einem interdisziplinären Symposium aus Anlaß des 60. Geburtstags von Fritz Oser (pp. 11–41). VS Verlag für Sozialwissenschaften.
Preacher, K. J., & Sterba, S. K. (2019). Aptitude-by-treatment interactions in research on educational interventions. Exceptional Children, 85(2), 248–264.
Prediger, S., Barzel, B., Hußmann, S., & Leuders, T. (2021). Towards a research base for textbooks as teacher support: The case of engaging students in active knowledge organization in the KOSIMA project. ZDM–Mathematics Education, 1–16. https://doi.org/10.1007/s11858-021-01245-2.
Rau, M. A., Aleven, V., & Rummel, N. (2017). Making connections among multiple graphical representations of fractions: Sense-making competencies enhance perceptual fluency, but not vice versa. Instructional Science, 45(3), 331–357.
Reinhold, F., Leuders, T., Loibl, K., Nückles, M., Beege, M., & Boelmann, J. M. (2024). Learning mechanisms explaining learning with digital tools in educational settings. A cognitive processes framework. Educational Psychology Review, 36(1), 14.
Renkl, A. (2014). Toward an instructionally oriented theory of example-based learning. Cognitive Science, 38(1), 1–37. https://doi.org/10.1111/cogs.12086.
Renkl, A. (2015). Different roads lead to Rome: The case of principle-based cognitive skills. Learning: Research and Practice, 1(1), 79–90.
Renkl, A. (2023). Exemplars. In R. Tierney, F. Rizvi, & K. Ercikan (Eds.), International encyclopedia of education (4th ed., pp. 612–622). Elsevier. https://doi.org/10.1016/B978-0-12-818630-5.14067-9.
Richter, J., Scheiter, K., & Eitel, A. (2016). Signaling text-picture relations in multimedia learning: A comprehensive meta-analysis. Educational Research Review, 17, 19–36.
Rittle-Johnson, B., Siegler, R. S., & Alibali, M. W. (2001). Developing conceptual understanding and procedural skill in mathematics: An iterative process. Journal of Educational Psychology, 93(2), 346.
Rittle-Johnson, B., Schneider, M., & Star, J. R. (2015). Not a one-way street: Bidirectional relations between procedural and conceptual knowledge of mathematics. Educational Psychology Review, 27, 587–597.
Roelle, J., & Berthold, K. (2016). Effects of comparing contrasting cases and inventing on learning from subsequent instructional explanations. Instructional Science, 44, 147–176.
Schalk, L., Schumacher, R., Barth, A., & Stern, E. (2018). When problem-solving followed by instruction is superior to the traditional tell-and-practice sequence. Journal of Educational Psychology, 110(4), 596.
Schwartz, D. L., & Martin, T. (2004). Inventing to prepare for future learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and Instruction, 22(2), 129–184.
Schwartz, D. L., Chase, C. C., Oppezzo, M. A., & Chin, D. B. (2011). Practicing versus inventing with contrasting cases: The effects of telling first on learning and transfer. Journal of Educational Psychology, 103(4), 759–775. https://doi.org/10.1037/a0025140.
Sinha, T. (2022). Enriching problem-solving followed by instruction with explanatory accounts of emotions. Journal of the Learning Sciences, 31(2), 151–198. https://doi.org/10.1080/10508406.2021.1964506.
Sinha, T., & Kapur, M. (2021). When problem solving followed by instruction works: Evidence for productive failure. Review of Educational Research, 91(5), 761–798.
Stigler, J. W., Gonzales, P., Kwanaka, T., Knoll, S., & Serrano, A. (1999). The TIMSS Videotape Classroom Study: Methods and findings from an exploratory Research Project on Eighth-Grade Mathematics instruction in Germany, Japan, and the United States. A Research and Development Report. US Department of Education. National Center for Education Statistics.
Sweller, J. (2010). Element interactivity and intrinsic, extraneous, and germane cognitive load. Educational Psychology Review, 22, 123–138. https://doi.org/10.1007/s10648-010-9128-5.
VanLehn, K. (1996). Cognitive skill acquisition. Annual Review of Psychology, 47(1), 513–539.
Weaver, J. P., Chastain, R. J., DeCaro, D. A., & DeCaro, M. S. (2018). Reverse the routine: Problem solving before instruction improves conceptual knowledge in undergraduate physics. Contemporary Educational Psychology, 52, 36–47.
Wittwer, J., & Renkl, A. (2008). Why instructional explanations often do not work: A framework for understanding the effectiveness of instructional explanations. Educational Psychologist, 43(1), 49–64.
Zhang, L., & Sweller, J. (2024). Instructional sequences in science teaching: Considering element interactivity when sequencing inquiry-based investigation activities and explicit instruction. European Journal of Psychology of Education, 1–11. https://doi.org/10.1007/s10212-024-00799-5.
Chi, M. T. H. (2000). Self-explaining expository texts: The dual processes of generating inferences and repairing mental models. In R. Glaser (Ed.), Advances in Instructional psychology (pp. 161–238). Lawrence Erlbaum Associates.
Acknowledgements
We would like to thank all members of the scientific network on PS-I (funded by the German Research Foundation) for the fruitful discussions.
Funding
Open Access funding enabled and organized by Projekt DEAL. This work was supported by the German Research Foundation under Grant number 439706396.
Open Access funding enabled and organized by Projekt DEAL.
Author information
Authors and Affiliations
Contributions
Conceptualization/idea: Loibl, Leuders. Analysis: Loibl, Leuders, Glogger-Frey. Writing - original draft: Loibl, Leuders; 3rd example: Glogger-Frey. Writing - review & editing: Loibl, Leuders, Glogger-Frey, Rummel. Figures: Leuders, Loibl. Project administration & funding: Loibl.
Corresponding author
Ethics declarations
Competing interests
The authors report there are no competing interests to declare.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Loibl, K., Leuders, T., Glogger-Frey, I. et al. CID: a framework for the cognitive analysis of composite instructional designs. Instr Sci (2024). https://doi.org/10.1007/s11251-024-09665-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11251-024-09665-9