Introduction

The Relevance of Self-regulatory Processes in Science Education

Not all classroom-based learning contexts demand the same level and complexity of learning. The thinking processes necessary for science learning are very different from the thinking involved in understanding other school subjects (Dillon 2008; Hodgson and Pyle 2010; Reif 2008). Reif argues that one of the main reasons students struggle with learning science is that they approach their learning as they would everyday knowledge, without appreciating the very specific and complex nature of science learning. In addition, researchers argue that students experience difficulty learning science due to the demands placed on them to independently accumulate vast amounts of knowledge (e.g. De Corte et al. 2004). This underlines the importance of science students developing abilities to independently control and monitor their learning.

These self-regulatory processes have become an important topic among educational and psychological researchers, principally because they have been found to enhance learning outcomes (Beishuizen and Steffens 2011). Empirical studies have shown the incidence of poor self-regulation in students today and its impact on academic achievement (Matthews et al. 2009). Researchers have found that the ability to self-regulate the learning process influences students’ goal setting (Adey 1992; Schunk 1990), increases their focus while performing academic tasks (Zimmerman 1990), and helps them assess their learning and the effectiveness of any strategies used (Cleary and Chen 2009). Self-regulatory and metacognitive processes are not only vital during school scholarship but are also life-long skills that learners can sustain after graduation and for self-education later in life (Abdullah and Lee 2007; Boekaerts 1997; Kaplan 2008; Kistner et al. 2010). Given the importance of life-long learning, which is at the forefront of both general and science-specific educational reforms (DfE 2013; Green 2003, 2011; Hodson 2003; Orrow-Whiting et al. 2007; Reiss et al. 1999), fostering self-regulated processes remains a primary focus of current research (Beishuizen and Steffens 2011; Dignath and Büttner 2008; Kistner et al. 2010; Zimmerman 2002).

The growing body of research regarding the benefits of developing students’ self-regulatory skills in the classroom has also extended to the implications for the field of science as a whole (Adey 1992; Driver 1989; Driver and Oldham 1986; White and Frederiksen 1998; Velayutham et al. 2012; Zohar 2004; Zohar and Dori 2012). As self-regulated learners in science have the ability to control and reflect on their learning, they are generally more motivated and personally interested in the material being studied, show increased academic performance, and are arguably more likely to provide greater contributions to current scientific knowledge (Velayutham et al. 2012; Zohar and Dori 2012). Together with the downward trends documented in the literature regarding student interest and motivation in science, most threatened between the ages of 10 and 14 years, these findings highlight the relevance of studying these processes in the science subject domain (Archer et al. 2010; Bennett and Hogarth 2009; George 2000; Ryan and Patrick 2001). The specific nature of science learning documented in the literature further reveals the complexity of this learning and highlights the importance of not only understanding the cognitive demands placed on students today in school science but also helping to support science students’ development and progression through the learning process.

The theoretical framework adopted for understanding these self-regulatory and metacognitive processes in the present study will now be presented before describing the intervention programme being investigated.

Framework for Understanding Self-regulatory Processes

From the social cognitive perspective, self-regulated learning (SRL) involves several interdependent phases through which learners manage their academic progression (Pintrich and De Groot 1990; Wolters 2010; Zimmerman 2002). One phase is commonly referred to as the forethought phase, which involves planning and setting goals and selecting strategies for a learning activity. During the monitoring phase, a student continuously tracks his or her progress and is aware of current performance in relation to the goals that were set. The activities involved in the control phase refer to implementing and acclimating learning strategies to complete the task. Finally, reviewing and responding to the learning experience makes up the reflection phase. In his framework, Pintrich (2004) details the self-regulatory activities involved in each of the phases in four separate areas: cognitive, motivation and affect, behavior, and context. The present study adopts this multidimensional framework for understanding SRL, and decisions regarding the measurement tools used in this study were guided by this conceptualisation. Velayutham et al. (2012) additionally highlight the importance of implementing strategies to develop self-efficacy and motivations when aiming to promote SRL in secondary school science. As such, the influence of the programme under study, the CREativity in Science and Technology (CREST) programme, on students’ beliefs toward their science learning was also investigated. Closely related to the topic of SRL is self-determination, which involves control, choice, and self-initiation of behavior (Glynn et al. 2009). This motivational aspect associated with SRL has been shown to be important in promoting autonomous learning, which helps students retain an intrinsic sense of learning and fosters the development SRL (De Bilde et al. 2011; Deci et al. 1991). Although not included in many studies of SRL among students, further insight may be provided through incorporating this construct into the understanding of student self-regulatory processes in the present paper. The science education intervention programme investigated will now be presented and positioned within this framework adopted for understanding these constructs among school students.

Understanding the CREST Programme Through a Self-regulatory Lens

Global curriculum reform efforts have been driven by a deeper appreciation of the scientific approach to learning and the importance of using inquiry strategies to understand the process behind scientific findings (Dillon 2008, Osborne and Dillon 2008). Recent science curriculum initiatives in the UK have resulted in the implementation of the British Science Association’s CREST award scheme. This inquiry-based intervention programme involves a 12-week science project (12 classroom hours—approximately 12 classroom sessions) for students between 11 and 14 years old and is offered to schools as a supplement to the UK science curriculum.Footnote 1 Led by students and facilitated by teachers, this programme focuses on promoting autonomy and peer collaboration and on providing students with opportunities to perform self-reflection and self-evaluation.

Before the CREST programme begins, teachers meet with a member of the CREST team, a mentor, to become familiar with the programme and obtain support regarding programme administration strategies. Similar to the intervention developed by Boekaerts (1997) aimed at fostering cognitive and motivational self-regulation, the teachers in the CREST programme are encouraged in this session to refrain from giving explicit procedural help to students, allowing them to reflect on their learning. This aspect of the programme may also address the concerns voiced by science educators and researchers through encouraging true science investigations and moving away from recipe-type structured activities (Dillon 2008; Lazarowitz and Tamir 1994; Tobin et al. 1994). Research support is rapidly growing for implementing inquiry-based learning opportunities in science classrooms, particularly in the laboratory context. However, while the use of laboratory activities in science classrooms is growing, documented by research reviews of the topic, there appears to be a lack of ‘true’ investigations (Abrahams and Reiss 2012; Lazarowitz and Tamir 1994; Tobin et al. 1994).

While laboratory investigations are aimed at developing higher-order cognitive thinking skills, research has documented that the majority of them are very prescriptive and that there is a tendency among educators to rely solely on recipe science experiments using lower-level skills as opposed to less guided, open-ended, student-led investigations (Dillon 2008; White and Frederiksen 1998). Almost two decades ago, researchers urged educators and administrators to move away from relying on lab manuals with prescribed guidelines and allow students to formulate their own procedures to test their hypotheses (Lazarowitz and Tamir 1994). The difficulty of conducting these open-ended investigation activities in the classroom setting may provide an explanation for the resistance seen in science classrooms today regarding the uptake of these suggestions. The CREST programme is structured in a way that allows the student-led projects to be initiated and conducted by the students, with teachers merely facilitating the learning. Researchers have also highlighted the importance of teachers being confident and having high efficacy for implementing new development strategies in their classrooms (Gaskill and Hoy 2002). By providing teachers with appropriate training and support, and structuring the materials to help students take control of their projects and science learning, the CREST programme may help make the classroom transition involved in implementing this new activity easier for the students and for the teachers.

The programme begins with teachers introducing students to CREST and allowing students to select groups of three to four peers to work with. During the initial session, student groups work together to develop a research question of interest to all group members. In cases where students do not agree, pupils are invited to explore other group interests more in line with their own and are able to move to a different group if necessary. All classes included in this study were mixed ability classes. In line with the principle aims of the CREST programme, students are encouraged to take on different roles and work to their own abilities. At this point, student-led discussions regarding how to work effectively in groups also take place. The CREST programme is structured in a way that allows the students to work together in self-selected groups toward personal goals while also reflecting on their learning in this environment. As the programme is explicitly presented to students as a chance to develop their teamwork skills, understanding the experience of students in the programme has extreme relevance for science education research. In a meta-analysis of self-regulated learning interventions conducted by Dignath and Büttner (2008), larger effect sizes were found for interventions that also contained an element of group work in the programme design. The research conducted by Urdan and Schoenfelder (2006) and Ryan and Patrick (2001) also documented that social aspects of the classroom environment and peer relationships can influence intrinsic motivation, especially for students transitioning from elementary to middle school.

However, while research has shown that working in groups can provide an environment which supports and promotes active reflection, evaluation, and monitoring during inquiry activities (Silver and Marshall 1990), educators cannot simply place students in science investigation groups and expect positive outcomes (Howe et al. 2007; Tobin et al. 1994). As with self-regulated learning development, students need to be supported in learning how to work collaboratively and develop the skill set necessary for these specific learning environments (Howe et al. 2007; Tobin et al. 1994). As the CREST programme is explicitly presented to students as a chance to develop their teamwork skills and effective collaborative learning strategies are discussed, the design of the CREST programme appears to be in line with research suggestions regarding developing students’ abilities to reflect, evaluate, and monitor their learning in science while also supporting teachers in implementing these activities.

Considering elements of the CREST programme within the context of self-regulated learning intervention research and connecting aspects of the programme to the theoretical framework described earlier provides support for the CREST award scheme as a viable pedagogical route through which to gain a better understanding of self-regulated processes and related motivations in young science students. Relating to the literature in this area, CREST aligns with the Self-Regulation Empowerment Programme (SREP) developed by Cleary and Zimmerman (2004) to foster self-regulated learning in students. Like the SREP, the CREST programme encourages students to set personal goals, monitor and reflect on their performance processes and outcomes, and make adjustments in order to manage independent projects (Cleary and Zimmerman 2004). The CREST programme also shares similar design elements with the Cognitive Acceleration Through Science Education (CASE) project in the UK centered on developing metacognitive skills in Maths, English, and Science (Adey 1992) and the Project to Enhance Effective Learning (PEEL) in Australia in secondary school science (Baird and Mitchell 1986). Like the above interventions, the CREST programme also directly aligns with De Corte et al.’s (2004) Competence, Learning, Intervention, Assessment (CLIA) framework for designing classroom environments that foster self-regulated processes. De Corte et al. (2004) identify cooperation among students, active knowledge construction, and self-direction as guiding principles for creating these environments. Therefore, while the programme is not explicitly aimed at developing these self-regulated processes among students, the similarities between CREST and targeted self-regulated interventions, as well as to the CLIA framework, are clear.

As mentioned above, during the initial sessions of the CREST programme, classroom teachers work with the students to explore areas of interest and support student groups in formulating a scientific question that they are personally interested in. By allowing students to come up with their own project hypotheses and methods, the programme introduces students to the investigative nature of science, addressing research concerns regarding the development of an appreciation of the nature of science among young students. As self-regulated learning within the framework outlined above involves goal-directed actions, thoughts, and feelings, providing students with opportunities to work toward the goals they have set for themselves and devise their own learning experiences as part of the CREST programme may also contribute to increases in self-regulated learning, particularly in the forethought stage (Boekaerts and Niemivirta 2000). As research has also shown that participating in open inquiry learning activities, giving students opportunities to be autonomous in their learning and have psychological freedom, can increase autonomous motivation and ownership for learning in science students, it is possible that CREST participation may also have this effect (Dillon 2008; Vansteenkiste et al. 2009). Providing students with opportunities for success and ensuring that students find tasks personally meaningful have also been shown to influence the development of self-efficacy and intrinsic motivation (Pintrich 2003; Schunk and Miller 2002). As students are provided with the opportunity to choose projects based on personal interests, CREST may also have a positive impact on the development of these related motivational constructs. Projects can be completed in Chemistry, Biology, or Physics classes and cover a range of topic areas, dictated by the students’ interests and often limited to school and teacher resourcing. For example, projects range from simple laboratory plant physiology experiments investigating nutrient requirements to complex chemical experiments exploring effect of pH on amylase activity.

Study Aim and Research Questions

The central aim of this study was to explore the immediate and longer-term impacts of the CREST inquiry-based learning programme on students’ and teachers’ self-reported levels of self-regulated processes and related motivations. The research questions and design were motivated by the findings of Moote et al. (2013) and relevant literature in this area, which will be discussed further in the following section. While this work focuses exclusively on self-regulation and related motivations, it is worth noting that the author acknowledges that the development of these constructs is only one aspect of successful science education. Three intervention conditions were included in the present study: the S1 CREST condition who participated in the programme during their first year of secondary school, prior to taking part in the study; the S2 CREST condition who participated in the programme during the study, in their second year of secondary school; and a No CREST condition who had no previous experience in the programme and who did not take part during the course of the study. Specifically, this study aims to address three sets of research questions:

  1. 1.

    Do students who participated in the CREST programme in their first year of secondary school prior to the study (the S1 CREST condition) have higher self-reported levels of self-regulated processes and related motivations and higher teacher-reported self-regulated learning compared to students coming into the study with no previous CREST experience (S2 CREST and No CREST conditions)?

  2. 2.

    Do students in the S2 CREST condition taking part in the CREST programme during the course of the study experience different changes in self-reported levels of self-regulated processes and related motivations immediately following programme participation, compared to students not taking part in the programme (S1 CREST and No CREST conditions)? Are similar changes observed on the teacher-reported measure?

  3. 3.

    Are any changes observed in students’ self-reports and reports from teachers retained 3 months following CREST programme participation for the S2 CREST condition?

Relating to the first research question, it was hypothesised that students in the S1 CREST condition would have higher pre-test self- and teacher reports than students in the other two conditions with no previous CREST experience. Relating to the second research question, it was hypothesised that students in the S2 CREST condition would experience positive changes in self- and teacher reports immediately following participation in the programme and that students in the other two groups would show no significant positive changes in the measured variables and may experience decreases in their self-reports (see Moote et al.). It was also hypothesised that teacher reports would increase from pre-test to post-test for students participating in the CREST programme during the study (S2 CREST condition) and that smaller changes may be noted in the two other groups. Finally, relating to the third research question, any positive changes in self-reports of the measured variables for the S2 CREST condition and any developments in teacher ratings of student self-regulated learning were expected to be retained 3 months following programme completion.

Extending Previous Work Relating to the Impact of the CREST Programme

As mentioned, the research questions and design of this study were informed by a similar quasi-experimental study conducted by Moote and colleagues which involved a sample of 73 students, 37 (51%) females and 36 (49%) males, aged 11–12. The findings of Moote et al. provided evidence of changes in students’ self-reported levels of self-regulated processes and related motivations after participating in the CREST inquiry programme. Specifically, Moote and colleagues documented the beneficial impact of CREST participation on student levels of self-regulated learning and self-determination. However, while this work incorporated delayed post-tests into the study design, the study was limited in that longitudinal data at pre, post, and delayed post-test was only available for a subset of students in the intervention condition. The research presented here builds on the findings of Moote et al. and also recommendations regarding the need to extend methodologies in the self-regulation domain beyond cross-sectional and immediate post-test designs (Berger and Karabenick 2011; Severiens et al. 2001; Zimmerman 2008).

Literature over the last two decades has highlighted the relevance of investigating changes in student processes and responses to tasks over longer periods of time (Duckworth et al. 2009; Pintrich 2003) as adaptations in response to environment specifically relating to self-regulation and motivation are evolutionary not instantaneous (Winne 1995). Changes in students’ levels of motivation and self-regulatory processes throughout the course of the school year, as well as the difficulties in retaining any developments in self-regulatory processes, provide additional support for the importance of understanding these constructs on a longer-term basis (Berger and Karabenick 2011; Throndsen 2011; Zimmerman 2008). Appreciating that long-term retention of self-regulated learning is essential in order to encourage and promote the transfer of regulatory processes to general life-long learning (Abdullah and Lee 2007; Boekaerts 1997; Kaplan 2008; Kistner et al. 2010) further highlights the need for research in this area. In addition, it can be argued that this transfer is especially important in science education with rapid advances in technology requiring new knowledge bases to be formed in individuals on an ongoing basis (De Corte et al. 2004; Duncan and Tseng 2010).

In view of the issues outlined above, a key focus of this study is on unpacking the longer-term impacts of the CREST programme on student self-regulated processes and related motivations. To enhance the research design of the work conducted by Moote et al. and provide more evidence regarding the impact of CREST, a number of methodological changes were implemented. Firstly, the key outcome variables were measured at three time points for all participants in the present study, contributing to the validity of the research findings obtained. While studies have documented increases in self-regulation following intervention programmes, several are limited due to the lack of appropriate control groups (Butler 1998, De Boer et al. 2012). As such, the present study includes two intervention conditions (S1 CREST who participated in CREST the previous year and S2 CREST who participated during the course of the study) in addition to a control group (No CREST), in order to provide further insight into the longer-term impact of the programme reducing threats to the external validity of the results.

Similar to other research domains, researchers studying self-regulatory processes in classroom learning environments have identified that multiple approaches greatly facilitate increasing the validity of research findings and their contributions to professional teaching practice (Matthews et al. 2009; Zimmerman 2008). Researchers have come to realise that the richest data is obtained from natural school settings and that interpretations from both teachers as well as students are important, valid components that should be included in analyses wherever possible (Kahle and Meece 1994; Matthews et al. 2009). Zimmerman and Martinez-Pons (1988) conducted research using teachers’ observations of students’ self-regulated learning performances in an attempt to validate a particular student self-report measure of the construct. These researchers proposed that the teacher observations provided a performance-based measure of the construct that can help to examine the validity of student self-report measures. Taking these suggestions into consideration, in addition to replicating the findings of Moote et al. and building on the longitudinal design, the present study also involved investigating teachers’ perceptions of the changes in levels of self-regulated learning for each student included in the study.

In summary, this study builds on previous work conducted by researchers in the field who have been developing our understanding of self-regulatory processes in the science classroom and builds on the findings of Moote and colleagues in order to provide a more complete picture of the impact of participation in an inquiry-based learning programme on self-regulated processes and related motivations among young science students. This study provides a distinct contribution to research, demonstrating that these constructs can be developed in natural classroom settings by promoting an environment that encourages students to be more self-regulated and motivated in their science learning. Through following a more rigorous experimental design involving two intervention conditions, collecting student and teacher data, and administering delayed post-tests to all participants, this study aims to also provide insight into the longer-term impacts of participation in the programme.

Method

Study Design

At the time of initial data collection, all students involved in the study were in their second year of secondary school and between 12 and 13 years old. While all students at the school participate in the CREST programme at some point during secondary school, participation in the programme was staggered. Of the 12 classes of students taking part in this study (the entire year-group from the school), four classes had taken part in the CREST programme the previous academic year (9 months before the pre-test was administered). These classes made up the first intervention condition (S1 CREST) included in the study design in order to investigate longer-term retention effects relating to the benefits of participation in the CREST programme. Another four classes were participating in the CREST programme during the course of the study and made up the second intervention condition for the analyses (S2 CREST). The final four classes included had no previous experience with the CREST programme in either their first or second year of secondary school and made up the control group included in the analyses (No CREST). This quasi-experimental study therefore followed a three-group (S1 CREST, S2 CREST, and No CREST) and three-phase (pre-test, post-test, delayed post-test) design producing a 3 × 3 mixed method design with the group variable being the independent measure and the change scores between the phase outcome measures being the dependent variables (pre-test/post-test change and pre-test/delayed post-test change).

Participants and Educational Context

Before any questionnaire measures were administered to students, school and parental consent were obtained following the ethical guidelines set by the British Psychological Society. Questionnaires were initially piloted with 20 students matching the target population (S2, 12–13 years of age), reviewed by two science teachers, and piloted for a second time with another 40 students. In an attempt to ensure that the programme was similarly implemented among classes, the piloting process also involved observing how teachers at the school implemented the programme with groups of students the previous academic year. These observations included recording the amount of time spent on the CREST programme, documenting the nature of teacher versus student control, observing the types of projects conducted, and discussing programme administration with the Head of Department.

To ensure that appropriate power was achieved, data were collected from an entire year-group of students (n = 240) from an independent (fee-paying private) school in Edinburgh in a high socio-economic catchment area. However, only students who completed both the pre-test and the post-tests were included in the analyses. Due to absences during class, only 194 students completed both pre- and post-test questionnaires.Footnote 2 Of the students who completed both questionnaires, 6 students did not finish the questionnaires in the allocated time leaving a total of 188 students (45% male, 55% female) in the study. All students involved in the study were judged by teachers to have adequate reading levels to work through the intervention materials. Table 1 shows a breakdown of the gender make-up and student numbers in the three groups. Over the course of 12 weeks, students in the S2 CREST condition worked on their CREST projects once a week, completing a total of 12 CREST sessions, each 55 min long (total hours on CREST ≅ 11 h).

Table 1 Student numbers and gender split with percentages for the three groups
Full size table

Pre-, Post-, and Delayed Post-test Measures

Student Measures

The three self-report measures in the field of self-regulation and motivation were chosen for this study, aligning with the framework for understanding the constructs of interest (discussed in Moote et al.). These included the Motivated Strategies for Learning Questionnaire (MSLQ; Pintrich and De Groot 1990); the modified Five Component Scale of Self-Regulation (FCSSR, MacLellan and Soden 2006); and the Science Motivation Questionnaire (SMQ, Glynn et al. 2009). A score for each subscale was generated by computing a mean for the items relating to each subscale. Tables 2, 3 and 4 present summaries of the subscales with example items making-up each scale (see Moote et al.).

Table 2 Example items and internal consistency (reliability) coefficients for the MSLQ subscales
Full size table
Table 3 Example items and internal consistency (reliability) coefficients for the FCSSR subscales
Full size table
Table 4 Example items and internal consistency (reliability) coefficients for the SMQ subscales
Full size table

Teacher Measure

While several measures are available in the literature to assess teachers’ perceptions of self-regulated learning as a construct and document teachers’ use of self-regulated learning within their classroom practices (Lombraerts et al. 2007; Mikroyannidis et al. 2012), few tools exists that measure teachers’ perceptions of self-regulated learning strategy use among their students. For the present study, these perceptions were assessed using the Rating Student Self-Regulated Learning Outcomes: A Teacher Scale (RSSRL) developed by Zimmerman and Martinez-Pons (1988). The RSSRL includes 12 items (presented in Table 5) relating to students’ use of self-regulated learning strategies easily observable by classroom teachers.Footnote 3 Items were rated along a 5-point scale (1 = never, 2 = sometimes, 3 = fairly often, 4 = very often, and 5 = always). The Cronbach’s alpha for the total scale calculated for the present study was .90, similar to the value of .92 published by Zimmerman and Martinez-Pons (1988). This provides support for using this measure and demonstrates excellent reliability for the published RSSRL scale.

Table 5 A summary the items for the RSSRL questionnaire
Full size table

Academic Performance

In order to control for prior academic performance and investigate its potential influence on students’ self-regulatory processes and related motivations, the present study included a measure of academic performance in science obtained before the study began. Pre-test academic performance measured by student marks on a science test completed by all students was included in the analyses. While it is appreciated that a more complete picture of assessment (including investigation/lab marks, daily quizzes, as well as presentation and homework marks in other science subjects) would be desired, the performance marks available were utilised. It was decided that this initial science test was sufficient to obtain a general sense of student attainment in science and contribute to the internal validity of this study.

Procedure

The pen-and-paper form questionnaires were administered in the classroom to all students before, after, and 3 months after students participated in the CREST programme. After the pre-test questionnaires were administered, the S2 CREST condition took part in the CREST programme over the course of 12 weeks while students in the other two conditions carried on with regular classes with no significant pedagogical events documented (including any major changes in teaching staff, curriculum structure, or school timetabling). Between the immediate post-test and delayed post-test, all students continued through the regular school term again with no significant pedagogical interventions or influential events noted. Copies of the teacher measure described earlier (the RSSRL) were distributed to the 12 classroom teachers (8 males, 4 females) while the students were completing the questionnaires. These questionnaires were administered to all 12 teachers within a week before, after, and 3 months after students in the second intervention condition (S2 CREST) completed the CREST programme.

Analysis

Parametric statistics were employed to provide the power required to answer the research questions and examine interactions between key variables. Before the parametric analyses were conducted, missing data analysis was performed. As there were no questions with more than 5% missing values, all questions on the questionnaires were included in composite scoring. The results from Little’s MCAR test for each of the questionnaires showed that any missing data was missing completely at random. At this point, it was deemed appropriate to use listwise deletion of cases for analysis and that no imputation was necessary. Multivariate analyses exploring gender differences were also conducted to determine whether gender should to be included as a covariate. As results revealed no main effects of gender on any of the variables, gender was not included as a covariate in the main analyses.

In order to address the first research question, one-way ANOVAs were run on all pre-test scores to investigate any differences between groups at the outset of the study. Repeated measures ANOVAs were avoided as the F test for treatment main effect is too conservative as pre-test scores are not affected by the intervention (Dimitrov and Rumrill 2003; Matthews et al. 2009). Therefore, following the suggestions of Dimitrov and Rumrill (2003) as well as Bonate (2000), one-way ANOVAs using the change scores of student and teacher self-reports while controlling for pre-test academic performance were conducted in order to address the remaining research questions of this study. To obtain a measure of change on each self-report scale, change scores were calculated by subtracting pre-test scores from post-test scores, as well as pre-tests from delayed post-test measurements.

As multiple measures were used for several constructs in this study (see Author 1 et al., anonymised for justification), data were analysed for these variables using MANCOVAs on the change scores described earlier. Running these multivariate ANOVAs is not only in line with the conceptual framework this research is based on but also reduces the chances of type 1 errors occurring by lowering the number of univariate ANOVAs conducted. For all multivariate tests reported, preliminary assumption testing was conducted to check for normality, linearity, univariate and multivariate outliers, homogeneity of variance-covariance matrices, and multicollinearity, with no serious violations noted unless otherwise stated. To simplify the analyses and make results more manageable, results will be presented in sections relating to the research questions outlined above. Table 6 presents an overview of the results relating to the three experimental condition and research questions.

Table 6 An overview of the results relating to the research questions and experimental conditions
Full size table

Results

Pre-test Group Differences and Retention Effects

Student Data

No statistically significant differences were found on the one-way between-groups ANOVAs conducted on all pre-test measures at the Bonferroni adjusted alpha level of .01. These results are not in line with the first research prediction that students in the S1 CREST condition who had previous experience with the CREST programme would show significantly higher self-reports on these measures. The means and standard deviations of these pre-test results are presented in Table 6.

Teacher Data

While an inspection of the means indicated that teacher ratings were highest for the S1 CREST group, the results from a one-way ANOVA on pre-test teacher ratings with group as the independent variable showed that the three groups were not significantly different on pre-test teacher ratings of self-regulated learning (F(2,181) = 1.462, p = .235, ηρ 2 = .0161). These non-significant results suggest no longer-term (9-month) impact of the CREST programme on teachers’ perceptions of student self-regulated learning. Considering these results together with the student data presented above provides further support for the lack of longer-term (9-month) retention effects in this sample. The means and standard deviations of these pre-test results are presented in Table 7 alongside the main variance analyses.

Table 7 A summary of the pre-test and immediate post-test means (standard deviations) of the three groups for all measures included
Full size table

Group Differences in Pre- to Post-test Change Using Student Data

Self-regulated Processes

A one-way between-groups (S1 CREST, S2 CREST, No CREST) MANCOVA was performed on pre- to post-test change scores, controlling for pre-test academic performance. Four dependent variables were used: MSLQ self-regulation, MSLQ cognitive strategies use, FCSSR total self-regulated learning, and SMQ self-determination mapping onto the theoretical framework adopted. There was a statistically significant difference between groups on the combined self-regulated process-dependent variables (F(8,272) = 2.392, Wilks’ Lambda = .873, p = .017, ηρ 2 = .066), and no significant multivariate test was found for pre-test academic performance. The result relating to academic performance suggests that lower-achieving students were no different in their changes in self-reported levels of self-regulated processes compared to higher-achieving students.Footnote 4

When considering the results for the four dependent variables separately, the only difference to reach statistical significance, using a Bonferroni adjusted alpha level of .01, was the self-regulation change score on the MSLQ (F(2,142) = 5.186, p = .007, ηρ 2 = .069). An inspection of the mean change scores indicated that both the No CREST and S1 CREST conditions decreased in self-reported levels of self-regulation (No CREST, M change = −.193, SD = .493; S1 CREST, M change = −.0786, SD = .639) while the S2 CREST condition increased (M change = .174, SD = .583). Further inspection of the 95% confidence intervals around each mean change score indicated that there was a significant increase in self-regulation for the S2 CREST condition only. These results are in line with the second research prediction suggesting that students taking part in the CREST programme, at the time of the study, experienced positive changes in their levels of self-reported self-regulation, while students in the other two groups not taking part in CREST showed decreases in these self-reports. The univariate tests relating to self-regulated learning, cognitive strategies use, and self-determination were not significant, therefore showing that the three groups did not differ in their self-reported changes on these measures from pre-test to post-test. Table 6 presents a summary of the pre-test, post-test, and change score means and standard deviations for these variables as well as for all outcome variables in this study.

Related Motivations

MANCOVA results investigating group differences in self-efficacy scores on the MSLQ and SMQ showed no statistically significant difference between the three groups on the combined dependent variables (F(4,350) = .413, p = .799, Wilks’ Lambda = .991, ηρ 2 = .005) while controlling for academic performance in science. Similarly, while controlling for academic performance at pre-test, there were no group differences in pre- to post-test change in intrinsic motivation scores on the MSLQ and SMQ combined dependent variables (F(4,344) = 1.612, p = .171, Wilks’ Lambda = .964, ηρ 2 = .018). Multivariate results investigating group differences in the test anxiety change scores on the MSLQ and SMQ also showed no statistically significant group differences on the combined dependent variables after controlling for pre-test academic performance (F(4,354) = .614, p = .663, Wilks’ Lambda = .986, ηρ 2 = .007).

Science-Specific Motivations

One-way between-groups ANCOVAs were performed to investigate group differences in the science-specific motivations measured on the SMQ. After adjusting for pre-test academic performance marks, there was no significant difference between the three groups on the career motivation change scores (F(2,187) = 1.979, p = .141, ηρ 2 = .021). In addition, no strong relationship was found between the change scores and the pre-test academic marks as indicated by the ηρ 2 value of .004. For overall science motivation, results showed that after adjusting for pre-test academic performance marks, no significant differences were found between the three groups (F(2,168) = .122, p = .885, ηρ 2 = .001). Similar to the analysis for career motivation, no strong relationship was found between pre-test academic performance and overall SMQ change scores (ηρ 2 = .003).

Group Differences in Pre- to Post-test Change Using Teacher Data

In order to investigate differences in teachers’ ratings of self-regulated learning between the three groups (S1 CREST, S2 CREST, and No CREST) before and immediately after participation in the CREST programme, a one-way ANCOVA controlling for pre-test academic performance was performed on the change scores of the teacher questionnaire results from pre-test to post-test. Results showed that Levene’s test of homogeneity of variance assumption was violated. However, ANCOVAs are robust to violations of this assumption provided that the ratio of the largest group variance is no larger than three times the smallest group variance (Field 2009, 2013). As this was the case for the present results, the ANCOVA approach was deemed appropriate.Footnote 5

The results from the above ANCOVA showed no significant differences between the three groups regarding the pre- to post-test change scores of teacher ratings of student self-regulated learning. In addition, no strong relationship was found between the covariate of academic performance at pre-test and the change scores, as indicated by the ηρ 2 value of .016. Table 7 presents a summary of these results, which are not in line with the research prediction that teacher ratings of self-regulated learning would increase the most for the S2 CREST condition. These results are also not in line with the increases seen from pre-test to post-test in the student data relating to self-regulation for the S2 CREST condition presented earlier while addressing the second research question.

Retention Effects of Group Differences Using Student Self-reports

Self-regulated Processes

The analysis reported earlier investigating pre- to post-test change using the student data was repeated comparing the change scores from pre-test to 3-month delayed post-test of the three groups (S1 CREST, S2 CREST, and No CREST) in order to investigate retention. For self-regulated processes, the multivariate results showed that there was a statistically significant difference between groups on the combined dependent variables after controlling for pre-test academic performance (F(8,236) = 2.497, Wilks’ Lambda = .850, p = .013, ηρ 2 = .078). When results for the dependent variables were considered separately, the only difference to reach significance using a Bonferroni adjusted alpha level of .01 was the self-regulation change score measured on the MSLQ (F(2,125) = 5.723, p = .004, ηρ 2 = .086). An inspection of the mean change scores indicated that both the No CREST and S1 CREST conditions decreased in self-reported levels of self-regulation (No CREST, M change = −.116, SD = .531; S1 CREST, M change = −.195, SD = .771), while the S2 CREST condition increased. Further inspection of the 95% confidence intervals around each mean change score indicated that there was a significant increase in self-regulation for the S2 CREST condition from pre-test to 3-month delayed post-test (M change = .162, SD = .730). The pre-test, delayed post-test, and change score means and standard deviations for all variables are presented in Table 8.

Table 8 A summary of the means (standard deviations) of the teacher self-regulated learning ratings for the three groups at pre-test and immediate post-test and the variance analyses results
Full size table

Related Motivations

Similar multivariate tests were run on the motivational measures using the delayed post-test minus pre-test change scores. MANCOVA results investigating group differences in the self-efficacy change scores on the MSLQ and SMQ showed no statistically significant differences between the three groups (S2 CREST, S1 CREST, and No CREST) on the combined dependent variables (F(4,320) = .498, p = .737, Wilks’ Lambda = .988, ηρ 2 = .006). Similar results were found on the multivariate tests for intrinsic motivation post-test to delayed post-test change scores on the MSLQ and SMQ, with no statistically significant difference between the three groups on the combined dependent variables (F(4,320) = .477, p = .775, Wilks’ Lambda = .989, ηρ 2 = .006). Multivariate tests run on the test anxiety change scores from pre-test to 3-month delayed post-test also showed no significant difference between groups (F(4,332) = .743, p = .563, Wilks’ Lambda = .982, ηρ 2 = .009).

Science-Specific Motivations

For the career motivation ANCOVA, after adjusting for pre-test academic performance marks, there was no significant difference between the pre- to delayed post-test change scores of the three groups (F(2,175) = .415, p = .661, ηρ 2 = .005) and no strong relationship was found between the change scores and pre-test academic marks. For overall science motivation, ANCOVA results showed that after adjusting for pre-test academic performance marks, no significant differences were found between the three groups (S2 CREST, S1 CREST, and No CREST) on total science motivation change scores on the SMQ (F(2,154) = .595, p = .553, ηρ 2 = .008) and no strong relationship was found between the covariate academic performance at pre-test and the pre- to delayed post-test change scores (ηρ 2 = .011).

Retention Effects of Group Differences Using Teacher Data

For the pre-test to 3-month delayed post-test change score analyses using the teacher data, the assumption of homogeneity of regression slopes was violated, and after inspection of the group variance ratios, it was decided that the covariate of pre-test academic performance would not be included in the one-way ANOVA conducted. Results showed no significant differences between the three intervention conditions on pre-test to delayed post-test change scores for teacher-rated self-regulated learning. Table 9 presents a summary of these results. Considering the results presented above relating to student self-reported changes in self-regulation from pre-test to three-month delayed post-test, it was expected here that teacher ratings in the S2 CREST condition would increase significantly while teacher ratings for the other conditions might decrease (No CREST and S1 CREST). As these trends were not found, the results presented here are not in line with the third research prediction.

Table 9 A summary of the pre-test and delayed post-test means (standard deviations) of the three groups for all measures included
Full size table

However, it should be noted that while no differences were found between the three intervention conditions in terms of changes in teacher ratings of self-regulated learning from pre-test to delayed post-test, significant increasing trends were found in all groups. In addition, from Table 9, it can be seen that the teacher perceptions of self-regulated learning increased the most for the S2 CREST condition. This result will be discussed regarding the changes in self-regulated learning from the teachers’ perspectives mirroring the student self-regulation trends presented earlier.

Relationship Between Teacher Ratings and Related Motivations

While the results presented thus far have investigated group differences in student and teacher self-reports on key outcome measures, additional insight can be gained by extending the investigation further to the relationship between teacher ratings and student self-reports on the other motivational variables measured. The relationships between teacher-rated self-regulated learning at pre-test and student pre-test self-reports on all outcome measures included were investigated using Pearson product-moment correlation coefficients. These analyses were repeated to investigate the relationships between teacher perceptions of self-regulated learning and student self-reports on all outcome variables at post-test as well as delayed post-test. Preliminary analyses were performed to ensure no violations of the assumptions of normality, linearity, and homoscedasticity. Partial correlations were also run to test the influence of controlling for pre-test academic performance on the relationships studied in each group with no significant differences from the zero-order correlations noted. Table 10 presents a summary of the above analyses, conducted firstly on the entire sample of students included in the study at pre-test, post-test, and 3-month delayed post-test (rows 1, 2, and 3) and then on each of the three groups of students at the three time points (S2 CREST = rows 4, 5, and 6; S1 CREST = rows 7, 8, and 9; No CREST = rows 10, 11, and 12). The correlations presented in Table 10 were also compared to the partial correlations described above, and an inspection of the zero-order correlations suggested that controlling for academic performance had very little effect on the strength of the relationships between the measured outcome variables presented.

Table 10 A summary of the means (standard deviations) of the teacher self-regulated learning ratings for the three groups at pre-test and delayed post-test and the variance analyses results
Full size table

From rows 1 and 2, it can be seen that no significant correlations were found between teacher rated self-regulated learning and student self-reports of self-regulated learning and self-regulation for all students included in the study at pre-test and post-test. These results provide further support for the findings presented earlier rejecting the second research prediction relating to the teacher data. However, significant correlations were found between teacher ratings of self-regulated learning and student self-reports of self-determination at pre-test, post-test, and delayed post-test for the entire sample. Significant correlations were also found between teacher ratings of self-regulated learning and student self-reports of self-efficacy, intrinsic motivation, and test anxiety. Results showed that teachers rated students higher on levels of perceived self-regulated learning when students had higher self-reports of self-determination, self-efficacy, intrinsic motivation, and reported lower levels of test anxiety. Therefore, while the results presented in this paper showed that changes in teacher ratings of self-regulated learning from pre-to post-test did not match the changes in student self-reports relating to self-regulated processes, they are in line with the changes observed on the related motivational outcome measures. The results for the delayed post-test correlations (rows 3, 6, 9, and 12) showed that teacher ratings of self-regulated learning were significantly correlated with all outcome measures.

Discussion

The results from Moote et al. (anonymised) indicated that the CREST programme led to measurable positive changes in students’ self-reported levels of some measured self-regulated processes and related motivations. This study aimed to replicate these findings in a different sample of students, provide more insight into the long-term retention of these changes, and also investigate whether these changes were observed from the teachers’ perspectives. These results will now be discussed relating to the three research questions.

Pre-test Group Differences and Longer-Term Retention Effects

While Moote et al. investigated the presence of retention effects 6 months after participation in the CREST programme, the present study explored whether retention was possible on a longer-term basis. As Moote et al. documented that developments in self-regulated processes and related motivations were maintained at the 6-month delayed post-test, it was predicted in the present study that students in the S1 CREST condition, with previous CREST experience 9 months before taking part in the study, would show higher self-reported levels on the outcome measures included than students in the other two conditions with no CREST experience (S2 CREST and No CREST). While an inspection of the trends showed that the S1 CREST condition had the highest pre-test scores on all self-regulated processes and motivational measures (with the exception of test anxiety), no significant pre-test group differences were found. Therefore, the results of the present study reject the first research prediction and suggest that any developments in self-regulated processes and related motivations in the S1 CREST condition (who participated in the CREST programme the previous year) were not retained.

The lack of longer-term retention effects reported are, however, in line with the findings documented by Montalvo and Torres (2004). Montalvo and colleagues suggest that it is necessary for students to have frequent opportunities to practice any developed self-regulated learning strategies through being given opportunities in the classroom in order to maintain them over time. Glaser and Brunstein (2007) also adopt this view and suggest that explicit self-regulation instruction is needed for lasting effects of writing interventions targeted at performance. Extending these results further, it might also be necessary for students to practice self-regulated strategies outside the classroom as research has shown that teachers and parents can foster autonomous motivation by giving students opportunities to be autonomous in their learning and have psychological freedom (Vansteenkiste et al. 2009).

As with the student self-report results, while the trends in the teacher data indicated that the group of students with previous CREST experience (S1 CREST) had higher teacher ratings of self-regulated learning, no significant group differences were found. These results therefore also reject the first research prediction relating to retention effects from the teachers’ perspectives. However, as with the student data results presented earlier, it is also possible that teachers’ perceptions did not change following CREST programme participation, and therefore, there were no developments to retain. Further research collecting more data from the S1 CREST condition is needed in order to generate stronger conclusions regarding this effect.

Group Differences in Pre- to Post-test Change Using Student Data

Analyses were also conducted to investigate whether students taking part in the CREST programme during the course of the study experienced significantly different changes at immediate post-test. Results showed that while the S2 CREST condition experienced significant increases in their self-reported levels of self-regulation, the other two groups decreased. These results are in line with the research prediction that students in the S2 CREST condition would experience positive changes in the outcome measures, providing support for the influence of the CREST programme on students’ self-reported levels of self-regulation at immediate post-test.

The above results contribute to the debate in current educational literature as to whether direct strategy instruction is necessary for the development of self-regulation in young students. Some researchers believe that strategy instruction needs to involve teachers modeling behaviours as well as explicitly explaining the strategies, how to use them, and what skills are required from the student (Boekaerts and Corno 2005; Hartman 2001; Kramarski and Michalsky 2009; Schraw et al. 2006; Zimmerman 2008). The results presented here, however, substantiate previous research findings on the other side of the debate, suggesting that curriculum-embedded self-regulated learning interventions can lead to improvements in self-regulation among students (Butler 1998; Perels et al. 2005); however, as discussed in the previous section, strategies relating to retention need to be considered. This study, along with the findings of Moote et al., suggests that by creating an environment in the classroom that fosters self-regulation, students can develop these learning processes further although more needs to be done to focus on maximising retention of these developments overtime.

The findings relating to the motivational constructs showed no significant differences between the three groups. These results are not in line with the significant changes observed in Moote 1 et al. or research that has documented increases in self-efficacy and intrinsic motivation in students following participation in interventions aimed at developing self-regulated learning (Fuchs et al. 2003; Stoeger and Ziegler 2010). However, De Corte et al. (2004) conducted a similar intervention study in mathematics with the fifth grade students who received the intervention from their classroom teachers and found only small effect sizes relating to related motivations. De Corte et al. (2004) suggested that even young students have years of experience in traditional classrooms that may need to be deconstructed before the potential motivational benefits of new environments for learning can be realised. In addition, the CREST programme presents a unique and challenging learning situation to students who may not be experienced in dealing with this amount of control for their learning. As a result, it is possible that students have low self-judgments of their abilities in science immediately after taking part in CREST and that any benefits related to improved self-efficacy and motivation would only be seen on the delayed post-tests. Results from the delayed post-tests showed that students in the S2 CREST condition, who participated in the programme during the course of the study, increased in their self-reported levels of self-efficacy 3 months after taking part in the CREST programme, while the other two groups decreased (S1 CREST and No CREST). It is therefore possible that participation in the CREST programme does influence the development of self-efficacy among students but that these benefits may take time to surface.

The analyses reported here also involved controlling for student academic performance marks at the beginning of the study, and no significant interactions were found for pre-test academic performance. In other words, the amount of change experienced on each variable measured was not related to student pre-test academic performance marks. Additionally, for the group of students taking part in CREST during the course of the study (S2 CREST), the benefit experienced (measured by change scores on the outcome measures) was not predicted by their academic performance marks at the beginning of the study. Replicating the findings of Moote et al., these results provide further support for the practical utility of the CREST programme, as students from a wide spectrum of achievers in science classrooms can benefit from participation and no disruption of the classroom make-up is necessary for this inquiry-based learning activity. These findings relating to academic performance also have implications in terms of implementation strategies for the CREST programme, as they reveal that the programme does not need to be targeted toward lower-achieving students.

Group Differences in Pre- to Post-test Change Using Teacher Data

The results from the analysis conducted on the pre- to post-test change scores of the teacher self-regulated learning, controlling for pre-test academic performance, showed no significant differences between the three groups immediately following CREST programme participation. This result was not in line with the research prediction that the group participating in the CREST programme during the study (S2 CREST) would have higher post-test teacher ratings of self-regulated learning than the other two groups not participating in the programme at the time of the study (S1 CREST and No CREST). The lack of significant correlations between post-test teacher and student ratings reported in Table 10 provides additional support for this finding. Therefore, while students reported that they were demonstrating higher levels of self-regulation in the science classroom immediately after taking part in the CREST programme (as presented in the previous results section), this development was not perceived by the teachers involved in the present study.

As students gain control in the classroom through conducting CREST investigations, which aim to answer a scientific question that is personally relevant to them, teachers need to facilitate this learning by supporting students in the development and utilisation of effective strategies. However, it is possible for an individual student to self-regulate toward his or her own criteria and personal goals, which may be different from the teacher’s set of perceived goals (Winne 1995). While the students may indeed be self-regulating, it is possible that they are not regulating the way in which teachers intend them to or in the particular direction to achieve the goals assumed by the teacher. This may explain the absence of a correlation between teacher and student self-reports of self-regulated learning at immediate post-test.

While the corroboration of teacher and student perceptions of self-regulated learning may have strengthened the validity of the measurement tools utilised, the lack of agreement between teacher and student reports of self-regulated learning provide important insight into the particular measurement tools employed and contribute to literary discussions regarding self-regulated learning theories. Different results from the teachers’ perspectives may also highlight the difficulty for teachers to measure and quantify internal processes such as self-regulation and motivation among their students while also exposing the difficulty for students to perceive and record these internal processes within themselves. These findings could also be explained considering research that documents the rigidity of teachers’ views toward students once they are formed, especially as teachers look for confirmatory evidence only to support or maintain this view (e.g. Carlone’s 2004 study of an advanced physics class, where teachers perceived male students as being ‘naturally brilliant’ and female students as ‘plodders’ despite evidence to the contrary).

Group Differences in Overall Change Using Student Data

Aligning with the third research prediction that any developments would be retained 3 months following CREST participation, the results showed that overall, students in the S2 CREST group experienced significant increases in self-regulation from pre-test to delayed post-test while students in the other two groups (S1 CREST and No CREST) experienced decreases throughout the course of the study. In other words, 3 months after taking part in the CREST programme, students in the S2 CREST group retained the higher levels of self-regulation they had developed through participating in the CREST programme.

In a meta-analysis looking at 95 studies of self-regulated learning in maths, reading comprehension, writing, and science in primary and secondary schools, De Boer et al. (2012) found only 17 studies that demonstrated long-term analyses of retention effects. The infrequency of studies reporting longer-term retention of intervention effects highlights the importance of the results presented in the present study. Among the studies that reported retention analyses, De Boer et al. (2012) discovered that maintenance effects were actually higher than the immediate post-test results in several of the self-regulated learning interventions included. The results here are in line with the findings documented by De Boer et al. (2012), as the effect size for self-regulation at 3-month delayed post-test (ηρ 2 = .078) was larger than at immediate post-test (ηρ 2 = .066).

Group Differences in Overall Change Using Teacher Data

The above analyses were repeated using teacher data and showed no group differences regarding changes in teacher ratings of self-regulated learning. Therefore, the overall significant increases seen from the student self-report results were not mirrored on the teacher ratings of self-regulated learning. However, teacher ratings did increase the most in the S2 CREST condition who experienced the programme during the course of the study.

As with the immediate post-test teacher results presented and discussed earlier, the different findings from the teachers’ perspectives may expose the difficulty for teachers to measure and quantify internal processes such as self-regulation among their students. These findings also highlight the difficulty for students to perceive and record these internal processes within themselves. As these results were more in line with the research predictions than the teacher findings at immediate post-test, one possible explanation for this finding is that teachers become more comfortable with the measure and are more able to match student perceptions as time goes on throughout the school year. However, through speaking with teachers in a post-study debrief, it was apparent that the teachers involved in the present study perceived students as maturing over the course of the academic year, and the results for teacher rated self-regulated learning may be due to this fact. Additionally, it is difficult to ask teachers to rate an instantaneous process when the measure used is relative to a recent time period.

Teacher Ratings of Self-regulated Learning and Related Motivations

The findings presented in Table 10 provide further support for the lack of agreement between the student and teacher data relating to changes in self-regulated processes from pre-test to post-test. However, while the results between teacher and student self-reports do not appear to agree relating to self-regulated learning and self-regulation in the science classroom, significant positive correlations were found between teacher ratings of self-regulated learning and student self-reports of self-determination, self-efficacy, and intrinsic motivation. One possible explanation for this finding could be that when teachers are judging self-regulated learning among their students, they may actually be looking at these related motivational variables. In other words, when teachers try to determine the extent to which students are regulating their own learning in the science classroom, they may actually be judging self-determination, self-efficacy, and intrinsic motivation. These results also demonstrate the possibility that these motivational variables are more externally visible to teachers compared to self-regulated learning, which is sometimes interpreted in the literature as an internal process. While part of self-regulated learning is the ability to monitor and regulate one’s surroundings, it is possible that it is too internal to be seen by teachers who are not trained as educational researchers studying observational methods for self-regulated learning. These correlational findings highlight the potential difficulty of using observational measures of student self-regulation in natural classroom settings by classroom teachers and the importance of training teachers to measure these constructs among their students.

In addition, the correlational results showed that at 3-month delayed post-test, the teacher and student self-report results were significantly correlated (Table 11). These results provide further support for the finding that the S2 CREST condition had the largest increase in teacher self-regulated learning ratings. In debriefing sessions with the teachers involved in this study, the possibility that teachers get to know students over the course of the year was discussed. This suggests that teachers become more familiar with what self-regulated learning entails through participation in the CREST programme and therefore report more occurrences of regulatory processes. It is also possible that students naturally progress through the academic year and develop self-regulated learning skills that teachers can see. However, the results relating to retention effects using the student data, which are in line with published research findings, did not demonstrate that students in all three groups increased in levels of self-regulated learning, making this explanation unlikely.

Table 11 Correlations between student and teacher outcome measures at pre-test, post-test, and delayed post-test
Full size table

Methodological Considerations and Future Research

The present study does not escape the limitations of quasi-experimental research conducted in natural classroom settings. In addition to the limitations regarding self-report measures (which assumes that the internal processes investigated are reasonably simplistic and amenable to measurement), the generalisability of the study results is limited to the particular school setting and student year-group included in the study. However, through replicating some of the trends reported in Moote et al. with an older sample of students, the generalisability of the results presented may be widened. As discussed in Moote et al., school contamination may also be an issue in the present study, as students in the two intervention conditions and the control group attended the same school. While the results discussed provide support for the influence of the CREST programme on students’ levels of self-regulation, it is not possible in the present study to unpack which aspects of the CREST programme contributed to the changes seen on the measured outcome variables in the present study. Similar issues have been documented in intervention studies regarding the difficulty in determining which aspects are necessary for improvements (De Corte et al. 2004; Glaser and Brunstein 2007; Williams and Williams and Binnie 2003). Further observational research conducting classroom observations (e.g. using the Fraser et al. 1995 instrument for assessing the science classroom environments) and multilevel quantitative analyses taking into account the clustered nature of the data and involving more comparison conditions is needed in order to fully address this issue. However, De Corte et al. (2004) state that the high degree of ecological validity is defensible and appropriate when looking at evaluating a classroom curriculum-embedded programme.

Considering that observer bias in the teacher measures of self-regulated learning has been documented recently in the literature (Matthews et al. 2009), appreciating that these biases may have affected the results presented in this study seems appropriate. Even further, as teachers’ post-test questionnaire responses were retrospective, it is possible that post-test results may have included their perceptions of the results of their implementation of the CREST programme (Rozendaal et al. 2005). In regard to the method that teachers used to measure the self-regulatory processes among their students, it is also possible that the teachers were rating students based on the norms of other students in a ranking process, which would also affect the validity of the results. In addition, the present study was not able to control for the level of ‘openness’ of the inquiry activities facilitated by the individual teachers. This may have also influenced the results presented for the different experimental conditions (S1 and S2 CREST). Therefore, observational data is needed to gain a deeper understanding of any related teacher effects. Further, the students involved in the present study were relatively young, so this additional observational data may also provide a deeper understanding of the level of ‘openness’ perceived among the individual student groups. A final limitation of this study is due to the small sample size of the 12 teachers involved. While the student data from each teacher is considerable, variation is expected between- and within-teacher measurements at the three different time points and may have contributed to the results presented in this paper.

This study has documented intriguing discrepancies between student and teacher self-reports of the impact of the CREST programme on self-regulation. Future research incorporating teacher data as a control in variance analyses in order to investigate which students are further away from the teacher measures may provide insight into this issue. The results of the present study have highlighted the difficulties of administering this measure to teachers and the importance of either developing teachers’ understandings of the constructs further before administering the questionnaires or having trained researchers make the observations directly.

Conclusion

While extensive research has been conducted on self-regulated processes and related motivations in students of all ages, the need for an increased understanding in natural classroom settings through implementing more rigorous research designs in specific learning contexts has been identified. Therefore, appreciating the interest seen in developing self-regulated learning and motivation in young students (e.g. Gläser-Zikuda and Järvelä 2008; Zimmerman 2002) and considering current discussions regarding the way science is taught around the globe (Kalman 2010; Leou et al. 2006), it was deemed important to explore the development of these constructs in young science students through participation in a curriculum initiative currently being implemented across the UK with over 30,000 students every year—the CREativity in Science and Technology (CREST) programme.

Through implementing a rigorous quasi-experimental research design using two intervention conditions and one control group with immediate as well as 3-month delayed post-test data, the results documented both the immediate and longer-term impact of CREST participation on students’ self-reported levels of self-regulation. Building on the work of Moote et al., the present study also aimed to explore changes in teachers’ perceptions of students’ self-regulated learning through CREST participation. By collecting data from the 12 teachers involved for each of their students at three time points, this study investigated the changes in student self-regulated learning from the teacher perspective, contributing to an area not yet explored in the literature. The group differences regarding changes in student self-reported self-regulation were not matched when looking at the teacher self-regulated learning results at both immediate post-test and delayed post-test. However, delayed post-test results indicated that the largest increase was found in the S2 CREST group that experienced the programme during the course of the study. By also investigating the correlations between teachers’ ratings of self-regulated learning and the other related student self-reported motivational variables measured, additional insight into the relationship between these variables and the potential difficulty for teachers to quantify self-regulated learning among their students was gained.

This study has provided further support for the practical utility of the CREST programme as a strategy to promote self-regulation among science students. The research presented has also addressed several issues highlighted in the literature as needing further research. First, this study investigated the retention of developed self-regulation among adolescent students specifically within a science inquiry-based learning context, exposed as an area needing more research attention. Additionally, as major career and educational decisions are made during the school years (Steffens et al. 2010), understanding the potential impact of strategies aimed at developing these self-regulated processes and related motivations on young students is an important contribution to knowledge in this field of educational research. The difficulty and complexity of creating environments that promote these processes among young students further highlights the importance of understanding how to appropriately support teachers in conducting these learning tasks with their students (Boekaerts and Niemivirta 2000).