Authentic learning in out-of-school labs
Given the theoretical model of authenticity in teaching and learning contexts developed by Betz et al. (2016), OSLs can be characterized as highly authentic learning settings that should—mediated by students’ perceived authenticity—provoke several positive effects in students. The model, which is illustrated in simplified form in Fig. 1, is divided into three successive parts. Part 1 highlights the characteristics of learners and of learning settings that may have an impact on students’ perceived authenticity of a learning environment. Part 2 outlines the interaction of these two components that results in students’ perceived authenticity of the learning environment. Part 3 summarizes the outcomes that may be affected by students’ perceived authenticity of the learning environment.
Building on this model, OSLs can be characterized as authentic learning settings, as several features of OSLs contribute to the authentic contextualization of students’ learning about scientific ways of thinking and working. Specifically, OSLs are often located at authentic research locations, such as universities, the instructors are often scientists or prospective scientists, and students usually use authentic materials (e.g. original data) and methods (e.g. inquiry-based methods) to work on authentic contents (e.g. current research questions or scientific problems) and open questions (i.e., innovation) (for more detailed characterizations of OSLs, see e.g. Garner & Eilks, 2015; Scharfenberg & Bogner, 2014). Thus, OSLs provide and emulate several features of authentic scientific practices. So far, to the best of our knowledge, eight studies have systematically investigated the effects of features hypothesized to determine the authenticity of an OSL setting on students’ motivational or cognitive learning outcomes. Table 1 gives an overview of these studies.
Most of the studies that are listed in Table 1 have examined the effects of the authenticity level of the location by comparing student learning in an OSL (i.e., authentic location) to learning in school (non-authentic location) on students’ motivational or cognitive learning outcomes. The results with respect to the cognitive effects (i.e., students’ knowledge acquisition and achievement) are highly inconsistent. While Scharfenberg et al. (2007) found a positive effect of the OSL setting on students’ knowledge acquisition, the findings from Itzek-Greulich et al. (2015) demonstrated no differences between the settings. The results of another study conducted by Itzek-Greulich et al. (2017) even showed negative effects of learning in an OSL compared to learning in school on students’ achievement. With regard to the motivational effects of the OSL setting, the findings are again not consistent. While the findings from Betz (2018) demonstrated a positive effect of the location on students’ situational interest (mediated by their perceived authenticity), the findings by Itzek-Greulich et al. (2017) and Itzek-Greulich and Vollmer (2017) mainly showed no differences between the learning settings with regard to several motivational outcomes, such as students’ situational interest. Also, the study by Schüttler et al. (2021) demonstrated no effect of the location alone on students’ situational interest. In their study, the location (i.e., the OSL) provoked only in combination with the provision of authentic learning material (i.e., high-tech lab equipment) positive effects on students’ situational interest (Schüttler et al., 2021). Schüttler et al. (2021) further showed that the students who worked with authentic materials reported the highest perceived authenticity, independent from the location. However, OSLs aim to complement and not to replace regular instruction and lessons in school (Garner & Eilks, 2015). Therefore, one could argue that the question on conditions for successful learning within OSLs becomes rather important than the question on the effectiveness of visiting OSLs compared to learning in school more generally.
In addition to the aforementioned study by Schüttler et al. (2021) that not only compared OSL-visits with learning in school but also investigated the role of authentic learning materials for learning within OSLs, three further studies have examined conditions for successful learning within OSLs: Scharfenberg et al. (2007), Mierwald et al. (2018), and Stamer et al. (2021). The latter investigated the impact of science videos showing regular practices of scientists on students’ perceived authenticity and their beliefs related to the work of scientists. The findings showed that students who performed nanotechnological experiments and watched the science videos reported higher perceived authenticity and developed more adequate beliefs about scientific practices than students who performed the same experiments but without watching the videos. (Stamer et al., 2021). Thus, the authenticity level of the learning materials had an impact on students’ perceived authenticity and their beliefs about science in an OSL. Mierwald et al. (2018) examined the effect of authentic learning materials on students’ epistemological beliefs about history science. Their results demonstrated that students who learned with original sources or with an audiobook of these original sources developed more sophisticated epistemological beliefs by the end of their OSL visit than students who learned with a rather traditional textbook. Thus, the authenticity level of the learning materials had an impact on students’ epistemological beliefs in an OSL. Scharfenberg et al. (2007) compared authentic hands-on experimentation to a (less authentic) learning activity without hands-on experimentation in an OSL for molecular biology. Their results showed no effect of the authenticity level of the learning activity on students’ knowledge acquisition.
It can be noted that the eight studies listed in Table 1 differ on various features, such as the investigated learning domains, the assessed outcomes, and the different effects of authenticity on motivational and cognitive outcomes. It can be further noted that only three studies (i.e., Betz, 2018; Schüttler et al., 2021; Stamer et al., 2021) assessed students’ perceived authenticity of the learning environment. This is surprising, as the very first investigations related to the effectiveness of OSLs focused on students’ perceived authenticity. One of the major findings by Engeln (2004), Glowinski (2007) and Pawek (2009), who evaluated particular OSL-projects for natural sciences, was that students perceived the OSL environment as authentic and that this perceived authenticity significantly predicted or correlated with students’ situational interest. A further study, conducted by Damerau (2012), also evaluated different OSL-projects for natural sciences and compared these projects to regular instruction in school (i.e., control condition without treatment). The results demonstrated a large effect of the three OSL-projects (compared to regular instruction in school) on students’ perceived authenticity but no effect on their situational interest (Damerau, 2012).
The association between students’ perceived authenticity and their situational interest (as demonstrated by Betz, 2018; Engeln, 2004; Glowinski, 2007; Pawek, 2009) is in line with both the authenticity model by Betz et al. (2016) and theories on the development of interest. The authenticity model implies the assumption that an authentically designed learning setting affects students’ perceived authenticity and in turn their situational interest or other outcomes (see Fig. 1). Theories on interest development describe situational interest as “a state or an ongoing process during an actual learning activity” (Krapp, 2002, p. 388) and assume that learners’ situational interest is first triggered and then maintained by external factors of the learning environment, such as meaningful and relevant learning activities (Hidi & Renninger, 2006). As mentioned above, the authentic contextualization of learning experiences aims at demonstrating the relevance and functionality of the targeted learning concepts to students. Hence, also in light of theories on the development of interest, it can be expected that an authentically designed and perceived learning setting fosters students’ development of situational interest. Furthermore, Lewalter and Geyer (2009) assume that visits of out-of-school settings, such as museums and science centers, promote students’ situational interest more generally, and that the authenticity—as one feature of these settings—demonstrates the relevance of the learning contents and thereby particularly evokes students’ maintained situational interest. However, their findings showed that students’ perceived relevance of the contents presented in a museum or science center strongly predicted both their triggered and maintained situational interest (Lewalter & Geyer, 2009). Against this background, it is reasonable to assume that students’ perceived authenticity of an OSL is associated with their (triggered and maintained) situational interest.
However, it is further surprising that, to our knowledge, only Scharfenberg et al. (2007) have investigated the effectiveness of an authentic learning activity (i.e., authentic hands-on experimentation versus an activity without authentic hands-on experimentation) for learning in an OSL. This is surprising, because the involvement of students in authentic learning activities that mimic scientific inquiry practices is a fundamental characteristic of OSLs (e.g. Euler, 2004; Pauly, 2012; Scharfenberg & Bogner, 2014). Engaging learners in inquiry-based learning activities is often also part of characterizations of authentic learning environments more generally (see Betz et al., 2016; Herrington & Oliver, 2000; Rule, 2006). But it has not yet been investigated whether students perceive activities that are assumed to simulate scientific inquiry processes as authentic and whether their perceived authenticity affects further outcomes, such as their situational interest. The present study aims to address this research gap by investigating the role of the authenticity level of a learning activity for students’ perceived authenticity and their situational interest in an OSL.
Authentic learning activities
The learning activities implemented in OSLs focus on “getting school students an authentic feeling for the scientific endeavor” (Euler, 2004, p. 26) and try to “place students in the role of being a scientist” (Glowinski & Bayrhuber, 2011, p. 374). The opportunities to situate learners in the role of a researcher by implementing respective learning activities differ depending on the discipline. OSLs for natural sciences focus on implementing hands-on activities that enable students to experiment with authentic tools and materials (e.g. Euler, 2004; Glowinski & Bayrhuber, 2011). OSLs for social sciences focus on implementing problem-based learning activities, which ask students to work independently on real-world and open-ended problems, in order to enable them to undertake research methods in these disciplines (Pauly, 2012).
In order to involve students in authentic activities that simulate features and processes of scientific inquiry, the PF (Productive Failure) activity seems particularly promising. PF asks students to develop conjectures about a complex and novel problem during an initial problem-solving phase before they receive instruction on canonical solutions (e.g. Kapur, 2015). Thus, PF students usually fail and struggle to canonically solve a complex problem. These failure experiences may be accompanied by uncertainty and frustration. Afterwards (i.e., during instruction), however, it is assumed that students discover the limitations of their previously posed conjectures and in turn learn from their failure (see Loibl et al., 2017). These processes demonstrate that PF students probably approach their understanding about a targeted learning concept just in the same way as scientists usually approach their knowledge about a given problem: Through failure and by discovering the boundaries of their current knowledge (see Firestein, 2015) and by falsifying their previously posed conjectures (see Chalmers, 2013). In addition, it is argued that PF emulates the scientific inquiry process of exploring solution ideas to a complex and novel problem without knowing whether one’s conjectures about potential solutions will be supported or not (see Cho et al., 2015). By emulating these major features and processes of scientific inquiry, PF holds the potential to situate learners in the role of a scientist (see Kapur & Toh, 2015) and can be characterized as an authentic learning activity. In previous studies, PF has typically been compared to DI (for a review, see Loibl et al., 2017). In contrast to PF, the DI activity can be characterized as less authentic. Due to students’ involvement in instruction first, followed by problem solving, DI approaches promote the conceptualization of scientific inquiry being nothing else but an application of instructions and thus a “simple, algorithm procedure” (Hodson, 1999, p. 784).
Besides emulating core features and processes of scientific inquiry, PF also fulfills several features that characterize authentic learning activities more generally. That is, several operationalizations of authentic learning activities (Betz et al., 2016; Herrington & Oliver, 2000; Rule, 2006) describe that authentic activities should involve complex problem-solving tasks that have to be independently and collaboratively investigated by learners through open inquiry. These features of authentic learning activities (i.e., complex problem, inquiry-based learning activity, collaboration between learners, and independent working) put forward by different theoretical models are in line with the design principles of PF (see Table 2).
As Kapur and Bielaczyc (2012) argue, the initial problem-solving phase of PF should match the following design principles: (1) the PF problem should be complex and open to various solutions (design principle: Problem/task), and (2) students should be able to explore and generate intuitive solution ideas to this problem by using their prior knowledge (design principle: Activity). (3) Students’ exploration activities during the problem-solving phase should be fostered by small-group collaboration (design principle: Participation structure) without instructional support from teachers or other facilitators, so that (4) students can independently explore solution ideas (design principle: Social surround). While PF matches several features of authentic learning activities, the problem-solving phase of DI fulfills none of these features (see Table 2). That is, the problem-solving task lacks complexity and novelty, as students usually only have to work on problems that are identical or isomorphic to the problem introduced during instruction (design feature: Problem/task). Due to the initial instruction phase, DI students also do not need to explore solution ideas, but rather have to apply and execute the problem-solving steps they were instructed on (design feature: Activity). Thus, they do not need to engage in a collaborative discourse about their shared solution ideas (design feature: Participation structure), and they do not have to independently develop and choose own problem-solving paths (design feature: Social surround). Therefore, also in light of common features of authentic learning activities, the PF learning activity seems to be more authentic than the learning activity of DI.
The previous study
As outlined before, numerous definitions and characterizations of OSLs and authentic learning settings more generally emphasize the role of certain learning activities in order to design for authenticity. Authentically designed learning settings are hypothesized to affect students’ perceived authenticity and, in turn, different motivational and cognitive learning outcomes (Betz et al., 2016; Gulikers et al., 2005) Research, however, has not yet focused on investigating whether students indeed perceive learning activities that are intended to be authentic as authentic and whether their perceived authenticity is related to further learning outcomes, such as students’ situational interest and knowledge acquisition. Against this background, we investigated the role of the authenticity level of learning activities in fostering students’ perceived authenticity and situational interest.
For this purpose, we compared the effects of PF as an authentic learning activity to DI as a less authentic activity on students’ perceived authenticity and their situational interest while learning the scientific practice of designing experiments in an OSL for social sciences (Nachtigall et al., 2018). Our results demonstrated that PF students did not report higher perceived authenticity nor higher situational interest than DI students. In addition, our findings showed that students’ perceived authenticity correlated with their situational interest but not with their knowledge acquisition in both conditions.
Based on these findings, we hypothesized that the authenticity level of other features of the learning setting (e.g. university campus, prospective scientist as an instructor, etc.), which were the same in both conditions, might have had a higher impact on students’ perceived authenticity than the authenticity level of the learning activity (i.e., PF versus DI). Specifically, due to a potential interrelatedness of the different features of an authentic learning setting, other authentic elements of the OSL (which were the same in both conditions) might have outweighed the effect of the authenticity level of the learning activity on students’ perceived authenticity and further outcomes.
Moreover, we assumed that school students may lack knowledge about research methods in the social sciences and especially about the notion that failure, struggle, mistakes, and uncertainty are an inherent part of scientific inquiry. Consequently, students may have difficulties to perceive the PF activity as more authentic than DI.
The present study
To address the aforementioned issues, we conducted a second study and implemented a slightly different design. We again compared the effects of PF with DI on students’ perceived authenticity and their situational interest while learning scientific practices in an OSL for social sciences. However, in the present study, we introduced various research methods within the social sciences, briefly described the steps of a typical research process, and communicated the notion that failure and mistakes are an inherent part of scientific inquiry to students in both conditions. Thereby, we aimed at addressing students’ potentially limited knowledge about social science research methods. We, moreover, selected a different learning topic: The scientific practice of evaluating causal versus correlational evidence. This topic was particularly appropriate in order to exemplify that researchers also make mistakes, such as interpreting correlational evidence as causal. To teach students the concepts of causality and correlation, the learning materials referred to a real study (i.e., a study that is published in a scientific journal) on the relation between children’s usage of media with violent content and their aggressive behavior. This content allowed for demonstrating that research can result in inconsistent, contradicting, and unclear evidence.
By using this slightly modified design in the present study, we aimed at replicating the findings of our first study. Specifically, we pursued what Schmidt (2016) calls a conceptual replication. Other than in a direct replication, we did not repeat the same procedure of our previous study but tested whether or not different learning materials and a slightly different procedure would lead to a repetition of our previous findings.
Research questions and hypotheses
As in our previous study, our major research question (RQ1) is whether the authenticity level of the learning activity (PF vs. DI) has an impact on students’ perceived authenticity and whether their perceived authenticity is associated with further outcomes, namely students’ situational interest and their performance on a knowledge test. Specifically, with respect to our RQ1, we derived and investigated the following four hypotheses:
Hypothesis 1 (H1)
PF students will report higher perceived authenticity at the end of their OSL visit than DI students. H1 builds on the argument that PF emulates features of authentic scientific inquiry and fulfills more features of authentic learning activities than DI (see Table 2).
Hypothesis 2 (H2)
PF students will report higher situational interest than DI students at the end of their OSL visit. H2 is based on theories on both authentic learning (e.g. Betz et al., 2016; Lepper, 1988) and interest development (e.g. Hidi & Renninger, 2006), which suggest that the authenticity level of the learning activity has an effect on students’ situational interest.
Hypothesis 3 (H3)
Students’ perceived authenticity will positively correlate with their situational interest, or the effect of the authenticity level of the learning activity on students’ situational interest will be even mediated by their perceived authenticity. H3 relies on the findings of previous OSL research (e.g. Betz, 2018) showing that students’ perceived authenticity correlates or predicts their situational interest.
Hypothesis 4 (H4)
Students’ perceived authenticity will positively correlate with their learning outcome (i.e., their performance on a knowledge test). H4 builds on the theoretical assumption that the authentic contextualization of learning fosters students’ understanding (e.g. Betz et al., 2016; Lepper, 1988). However, it should be mentioned that PF students in the present study did not outperform DI students on a knowledge test (see our parallel study on the same dataset: Nachtigall et al., 2020). However, the findings of our previous study demonstrated no correlation between students’ perceived authenticity and their performance on a knowledge test. Thus, it is interesting to examine whether the theoretically assumed association between students’ perceived authenticity and their knowledge acquisition can be empirically supported, and whether or not the present study replicates the findings of our previous study.
In addition to testing these four hypotheses, we aim to explore whether students perceived authenticity of the learning activity (i.e., PF versus DI) is related to their perceived authenticity of other features of the learning setting (RQ2). For this purpose, we administered an additional instrument in the present study (which was not used in our previous study) for assessing students’ perceived authenticity of different features of the learning setting.