Advertisement

Asia Pacific Education Review

, Volume 17, Issue 2, pp 187–202 | Cite as

The effectiveness of self-regulated learning scaffolds on academic performance in computer-based learning environments: a meta-analysis

  • Lanqin ZhengEmail author
Article

Abstract

This meta-analysis examined research on the effects of self-regulated learning scaffolds on academic performance in computer-based learning environments from 2004 to 2015. A total of 29 articles met inclusion criteria and were included in the final analysis with a total sample size of 2648 students. Moderator analyses were performed using a random effects model that focused on the three main areas of scaffold characteristics (including the mechanism, functions, delivery forms, mode, and number of scaffolds; how to promote self-regulated learning by scaffolds); demographics of the selected studies (including sample groups, sample size, learning domain, research settings, and types of computer-based learning environments); and research methodological features (including research methods, types of research design, types of organization for treatment, and duration of treatment). Findings revealed that self-regulated learning scaffolds in computer-based learning environments generally produced a significantly positive effect on academic performance (ES = 0.438). It is also suggested that both domain-general and domain-specific scaffolds can support the entire process of self-regulated learning since they demonstrated substantial effects on academic performance. Different impacts of various studies and their methodological features are presented and discussed.

Keywords

Self-regulated learning Academic performance Scaffold Meta-analysis 

Introduction

Self-regulated learning (SRL) is regarded as one of the most important competences for lifelong learning (Ifenthaler 2012). It is also the key component for mediating success in most learning environments (Hartley and Bendixen 2001; Lehmann et al. 2014; Zimmerman 2008). The number of studies focusing on SRL has increased rapidly in the past decades. At the same time, the research focus has shifted from the concept of SRL to the applications of self-regulation (van Ewijk 2011). In recent years, more and more researchers have expanded their interests to the promotion of SRL through a range of methods (Dignath and Büttner 2008). These methods include, for example, regulating learners’ motivation and emotion (Efklides 2011) or directly teaching learning strategy (Kistner et al. 2010) in different learning domains. In addition, many researchers have developed scaffolds to facilitate SRL in computer-based learning environments (Azevedo et al. 2004; Dabbagh and Kitsantas 2005; Delen et al. 2014). However, there is still a lack of evidence concerning the effectiveness of SRL scaffolds (Azevedo et al. 2004; Devolder et al. 2012). The current study aims to investigate the effectiveness of computer-mediated SRL scaffolds through an in-depth meta-analysis.

Evidence from the literature shows that high-achieving students have distinct self-regulatory abilities, such as accurate monitoring of learning and adaptation of metacognition (Isaacson and Fujita 2012). There is also agreement on the effectiveness of SRL on academic performance based on numerous empirical studies (Chung 2000; Chen and Huang 2014; Paris and Paris 2001; Zimmerman and Bandura 1994). However, little is known about which aspects of SRL intervention have resulted in the positive impact on academic performance. In this study, we focus on the effects of different kinds of SRL scaffolds on academic performance in computer-based learning environments. The meta-analysis approach was adopted to investigate the effectiveness of SRL scaffolds on academic performance in computer-based learning environments to gain an overview of the evidence on the effects of SRL scaffolds.

Meta-analysis is the quantitative approach to systematic review, and it has been widely used in different fields (Bangert-Drowns and Rudner 1991). An earlier study by Kulik and Kulik (1989) demonstrated that meta-analysis was very useful in gathering information about the effectiveness of interventions. Furthermore, meta-analysis is more systematic than other kinds of reviews (Borenstein et al. 2011), which can help to determine the most effective interventions from a range of options. Importantly, meta-analysis can transform the outcomes of different studies into a common metric, which allows to compare the results of different studies using different instruments and interventions (van Ewijk 2011). In addition, Hedges and Olkin (1986) reported that the overall effect of different studies obtained through meta-analysis was more accurate than the individual studies. However, a meta-analysis of computer-mediated SRL scaffolds remains lacking. This meta-analysis aims to review empirical studies on computer-mediated SRL scaffolds from 2004 to 2015. The following section provides an overview of SRL scaffolds in computer-based learning environments.

Literature review

Self-regulated learning in computer-based learning environments

SRL is defined as an active and constructive process through which learners can set goals, and monitor and control their cognition, motivation, and behavior (Printrich 2000). It is also characterized as a self-directive process as self-beliefs enable learners to transform their academic abilities (Zimmerman 2008). Winne and Hadwin (1998) proposed that SRL included four phases: defining the task, setting goals and plans, enacting tactics, and adapting metacognition. Therefore, learners need to analyze the learning context and define tasks, set the appropriate learning goals and make plans, select the effective learning strategies to use, monitor the whole learning process, and evaluate their learning performance. Based on the evaluation, they may need to modify and adapt certain aspects of their motivation, emotion, cognition, and metacognition (Azevedo et al. 2009).

Previous studies indicated that SRL is the crucial skill for success in computer-based learning environments (Adeyinka and Mutula 2010). However, learners cannot always regulate themselves successfully because of reasons such as lack of good strategy use (Pressley and Hilden 2006), lack of metacognitive knowledge (Veenman et al. 2006), failure to control of metacognitive processes (Hacker et al. 2009), or lack of experience in learning environments with multiple representations (Azevedo et al. 2009). Thus, how to foster SRL ability has become a central issue in the field of education research and practice.

Azevedo et al. (2004) proposed that computer-based learning environments present the promising potential to foster students’ abilities to regulate their learning. Computer-based learning environments provide important opportunities for self-directed learning and allow for a high degree of learner control (Lajoie and Azevedo 2006; Winters et al. 2008). Computer-based learning environments are typically characterized by multiple representations of information that can be controlled by the learner and can help to define tasks, set goals, enact strategies, and adapt metacognition (Winters et al. 2008). Researchers have explored and developed various kinds of computer-based learning environments to scaffold SRL. For example, Perry and Winne (2006) developed gStudy software to capture students’ self-regulated engagements using time-stamped trace data. Azevedo et al. (2010) designed a hypermedia learning environment, Meta Tutor, to detect, trace, and foster SRL about human body systems through pedagogical agents. Narciss et al. (2007) used non-embedded indirect intervention tools for monitoring and evaluation of learning tasks to promote SRL. In addition, assessment of self-regulation can be supported through computer-based learning environments. Aleven et al. (2010) proposed a method for context-sensitive assessment of SRL strategy with an intelligent tutoring system. SRL scaffolds in computer-based learning environments have become increasingly important because students cannot master higher-order thinking without such guidance (Simons and Klein 2007).

Self-regulated learning scaffolds

It is widely acknowledged that the use of scaffolds can support SRL processes (Dabbagh and Kitsantas 2005; Shapiro 2008). Scaffolds can be defined as the process through which learning efforts are supported in open learning environments (Hannafin et al. 1999). Scaffolds can be tools, strategies, prompts, metacognitive feedback, or guides that help students to achieve higher levels of understanding beyond their current abilities (Jackson et al. 1994; Simons and Klein 2007). The theoretical underpinning of scaffolding is that it can provide guidance for students on an as-needed basis and fade as abilities increase (Hogan and Pressley 1997).

Scaffolds can be categorized based on different functionalities. Hannafin et al. (1999) classified four kinds of scaffolds, namely conceptual, metacognitive, procedural, and strategic. Conceptual scaffolds aim to provide guidance about what to consider when a problem is already defined (Hannafin et al. 1999). Metacognitive scaffolds include non-human or human agents who can help students self-regulate the underlying problem-solving process (White et al. 2000). Procedural scaffolds can provide guidance about how to use resources or perform tasks (Azevedo et al. 2001). Strategic scaffolds provide different techniques or methods as well as solution paths for students (Lajoie et al. 2001). Scaffolds can differ in their delivery mode and may be embedded or non-embedded, fixed or adaptive, hard or soft, direct or indirect (Devolder et al. 2012). Embedded scaffolds are integrated in the learning environment so that students are obliged to pay attention to them, while non-embedded scaffolds are initiated by students themselves (Narciss, Proske, and Koerndle 2007). Fixed scaffolds are static and will not adapt to individual students’ needs, while adaptive scaffolds can adjust to meet students’ needs (Azevedo et al. 2004). Hard scaffolds are static or fixed, and soft scaffolds are customizable and negotiable (Saye and Brush 2002). Direct scaffolds are explicit, while indirect scaffolds are tacit (Stahl and Bromme 2009).

To date, however, there is no agreement on the effectiveness of SRL scaffolds. For example, studies revealed that prompts were effective in terms of regulation of student cognition (Lee et al. 2010; Devolder et al. 2012). Moos and Azevedo (2008) found guiding questions that can help students to be aware of and monitor their learning were effective scaffolds.

Concept mapping tools can also serve as effective scaffolds in promoting SRL (Marée et al. 2013). Therefore, researchers designed and developed different kinds of tools using advanced technologies to scaffold SRL. For example, Poitras and Lajoie (2014) developed an agent-based adaptive system for scaffolding SRL in history education by tracing techniques. Lee et al. (2010) found that a combination of generic learning strategy prompts with metacognitive feedback can promote learners’ recall and comprehension in a computer-based learning environment. Lehmann et al. (2014) examined the effectiveness of preflective as well as reflective prompts and indicated that directed preflective prompts worked best for novice learners in an online learning environment.

In summary, there is consensus on the need for scaffolding to promote SRL in computer-based learning environments (Devolder et al. 2012). However, there is still a lack of clarity about the most effective type of scaffold in terms of supporting SRL processes. The present study provides an overview of the effectiveness of SRL scaffolds in computer-based learning environments through the meta-analysis approach.

Findings from earlier meta-analyses of SRL

Meta-analysis can compare the results of different studies and provide the overall effect through one common metric (van Ewijk 2011). To date, meta-analysis of SRL remains limited, and only a few meta-analyses have examined the effectiveness of intervention in the area of SRL. An early meta-analysis by Hattie et al. (1996) aimed to identify features of study skills interventions in 51 studies. Their findings revealed an average effect size of 0.57 for academic performance, 0.16 for study skills, and 0.48 for affect. Later, Benz (2008) investigated the relationship between SRL and academic performance and found the overall effect of interventions on academic performance was 0.78.

Dignath et al. (2008) meta-analyzed 30 studies on fostering SRL at the primary school level and found that SRL training programs are effective. Dignath and Buettner (2008) investigated the impact of different training characteristics on the outcomes by the meta-analysis of 49 studies conducted among primary school pupils and 35 studies with secondary school students. Their results indicated an overall mean effect of 0.69 with no significant difference between the two samples in terms of the mean effects. Sitzmann and Ely (2011) examined how adults regulated their learning of work-related knowledge and skills, and found that goal level, persistence, effort, and self-efficacy were the self-regulation constructs with the strongest effects on learning.

These earlier meta-analyses contributed to insight into the effectiveness of SRL intervention. However, they also highlighted some methodological problems such as studies with no control group, and where initial equivalence between experimental and control groups had not been established (Sitzmann and Ely 2011). In addition, the previous meta-analyses did not clarify the effectiveness of SRL scaffolds on academic performance in computer-based learning environments. Moreover, it was noted that few studies provided empirical evidence about the type of scaffolds that affect academic performance. Therefore, the present meta-analysis was performed to identify and examine SRL scaffolds and their effects, demographics of the selected studies and methodological features. We aimed to overcome the methodological problems and to investigate the following research questions:
  1. 1.

    Which kinds of SRL scaffolds (including mechanism, functions, delivery forms, mode, numbers, and manner in which SRL is facilitated by scaffolds) are the most effective in improving academic performance in computer-based learning environments?

     
  2. 2.

    What study characteristics (including sample groups, sample size, learning domain, research settings, and types of computer-based learning environments) moderate the effects of SRL scaffolds on academic performance?

     
  3. 3.

    What methodological features (including research methods, types of research design, types of organization for treatment, and duration of treatment) moderate the effects of SRL scaffolds on academic performance?

     

Methodology

Meta-analysis procedure

The present study follows the seven steps of meta-analysis proposed by Cooper (2010): (1) define research questions; (2) formulate the selection criteria to include and exclude studies; (3) search the literature; (4) code studies based on the scheme; (5) analyze the effects of interventions; (6) analyze the moderating effects of study characteristics; (7) draw final conclusions and results of the meta-analysis.

Paper selection processes

This study selected papers relevant to SRL scaffolds in computer-based learning environments in the Web of Science database from 2004 to 2015. Previous studies have suggested the Web of Science database as the most popular measure of the quality of the scientific publications quality (Casal et al. 2011). As the study aimed to analyze the evolution of computer-mediated SRL scaffolds in the past decade, the time frame 2004–2015 was selected. The paper selection process proceeded in two steps. In the first step, 450 papers closely related to SRL in the field of educational research were selected using paper title and keywords search (self-regulated learning, SRL, self-regulated learning scaffold) within the Web of Science database. For the second step, the authors scanned the full text of each paper to decide which ones were suitable for inclusion based on the following criteria:
  1. 1.

    Only research articles were included in this study. Book reviews, book chapters, proceeding papers, meeting abstracts, editorials, news, and reviews were excluded.

     
  2. 2.

    The scaffolds should be computer-mediated and focus on SRL processes and academic performance. Human or peer scaffolds were excluded.

     
  3. 3.

    Only experimental or quasi-experimental studies related to the effects of scaffolds on academic performance were included. Conceptual studies and studies not reporting academic performance were excluded.

     
  4. 4.

    The studies need to provide data to calculate effect size (ES), including means and standard deviation, F or t values.

     
  5. 5.

    Studies could have taken place in any country but needed to be reported in English.

     
  6. 6.

    Studies should include control groups and provided pretest data.

     
  7. 7.

    Studies could include students in K–12 and in higher education.

     

Of 450 papers, only 88 articles met the first three criteria. After rigorous selection by scanning the full text, only 29 articles met the remaining criteria. Greenhalgh and Peacock (2005) indicated that snowball sampling was another useful search strategy. After searching the reference lists of the 29 articles, it was found that no further papers met the criteria. The present study thus included 29 journal papers indexed in the Social Sciences Citation Index of the Web of Science, which were analyzed further. The main reason for this small number was that computer-mediated scaffolds were not always available, or measures of academic performance or the statistical information required for calculating effect size were not always reported.

Coding scheme

Three main study characteristics were examined in detail, including characteristics of the scaffolds, demographics of the selected studies, and methodological features. The scaffold characteristics included the scaffolds’ mechanism, function, delivery form, mode, numbers of scaffolds, and how to facilitate SRL by scaffolds. Demographics of the studies were analyzed across the research sample groups, sample size, research learning domain, research settings, and types of computer-based learning environments (CBEs). Methodological features included research methods, types of research design, types of instruction, and duration of treatment. Table 1 illustrates the coding schemes of each sub-dimension in detail. Some of these sub-dimensions, namely research sample groups, research settings, research learning domains, research methods, types of research design, and types of instruction for treatment, have also been applied in other reviews (Kinshuk et al. 2013; Wang et al. 2014) or meta-analyses (Chang and Lin 2013).
Table 1

Coding schemes for each dimension

Super-dimensions

Sub-dimensions

Coding scheme

Scaffold characteristics

Mechanism of scaffolds

Prompts or hints; concept map; integrated self-regulated learning tool (including goal setting, strategic planning and enacting, self-evaluation, and monitoring); worked example

Functions of scaffolds

Conceptual (e.g., draw the important concepts and link them); strategic (e.g., using the next 10 min for reflection); procedural (e.g., what will be your next step?); metacognitive (e.g., please monitor your learning processes); multiple

Delivery forms of scaffolds

Direct; indirect

Mode of scaffolds

Domain-specific scaffolds (e.g., what is the trend of solar radiation?); domain-general scaffolds (e.g., stop and reflect)

Number of scaffolds

Multiple; unique

How to facilitate SRL by scaffolds

Support monitoring; support adaptation; supporting the entire process of SRL

Characteristics of the selected studies

Sample groups

Primary school; junior and senior high school; higher education

Sample size

20–50; 51–100; 101–300

Learning domain

Natural science (including science, mathematics, physics, biology, geography); social science (including politics, education, psychology, linguistics,); engineering and technological science (including engineering, computer science, educational technology); medical science (including health and medicine); mixed learning domain

Research settings

Laboratory; classroom; distance learning settings

Types of CBEs

Client–server-based hypermedia learning environment; web-based learning environment; concept mapping tool; mobile learning environment; agent; inquiry learning environment

Methodological features

Research methods

Quantitative methods; qualitative and quantitative methods

Types of research design

True experimental design; quasi-experimental design; non-experimental design

Types of organization for treatment

Individual; group

Duration of treatment

Less than 1 h; 1–10 h; 1–7 day; 2–4 weeks; 5–8 weeks; 9–24 weeks

Inter-rater reliability

Two raters independently coded all the studies based on the coding schemes. The agreement rate calculated by Cohen’s kappa was 0.88, which was regarded as reliable (Cohen 1960). The two raters discussed and resolved all discrepancies.

Analysis

To calculate the effect sizes, we adopted the software Comprehensive Meta-Analysis (CMA), developed by Biostat (www.meta-analysis.com). In the present study, moderator analysis was conducted to examine the direction of the differences between subgroups and between the average effect sizes of the variables (Karadağ et al. 2015). The effect size d was calculated by the standardized mean difference, which was equal to the difference between the mean values of experimental and control groups divided by the standard deviation (Hedges and Olkin 1986). Since the outcome variable was academic performance, we only calculated the effect sizes of academic performance. If one study adopted multiple tests to examine academic performance, these effect sizes were averaged to obtain one representative effect size for each study by CMA software. Since a random effects model allows sampling error at the study level (Dignath et al. 2008), a random effects model was adopted to report the effect size, standard errors, confidence intervals, and z and p values in the present study. In addition, the publication bias and sensitivity analyses were conducted by CMA to examine the analysis results.

Results

Overall effect

Hattie’s (2009) definition of effect size was adopted, with an effect size of 0.2 considered as small, 0.4 as medium, and 0.6 as large. Table 2 shows the number of articles (k), effect size (ES), standard error (SE), variance, confidence intervals, z value, p value, and test of heterogeneity in effect size for all articles. The overall weighted effect size was 0.438 for 29 articles with a total sample size of 2648 subjects. The meta-analysis results revealed that SRL scaffolds had a medium but significantly positive effect on academic performance. This effectiveness has been confirmed by the overall weighted effect sizes of the 29 articles. Furthermore, generalization of these results is regarded as very stable and reliable because 2648 subjects were involved in the meta-analysis. The result of the heterogeneity test indicated that the distribution of effect sizes among the articles was significantly heterogeneous (Q = 408.224, df = 28, p = 0.000). This implies that some factors other than sampling error accounted for the variance. Table 3 shows the effect size (ES), standard error (SE), variance, confidence intervals, and z and p values for each study. In the next section, the characteristics of scaffolds, demographics of the studies, and methodological features were examined to investigate the moderating effect.
Table 2

Overall effect sizes

 

k

ES

SE

Variance

95 % CI

Z

p

Test of heterogeneity in effect size

Lower

Upper

Q

df (Q)

p

Fixed

29

0.338

0.027

0.001

0.281

0.387

12.383

0.000

408.224

28

0.000

Random

29

0.438

0.107

0.011

0.229

0.648

4.096

0.000

   
Table 3

Sample size, effect size (ES), standard error (SE), variance, confidence intervals, Z values, and p values for all studies

Author(s) (Year)

N

ES

SE

Variance

95 % CI

Z

p

Lower

Upper

Azevedo et al. (2004)

51

0.165

0.140

0.020

−0.110

0.440

1.174

0.240

Azevedo and Hadwin (2005)

111

0.074

0.116

0.014

−0.154

0.302

0.638

0.523

Bannert and Reimann (2012)

80

0.079

0.129

0.017

−0.174

0.332

0.612

0.540

Bannert et al. (2015)

70

0.057

0.138

0.019

−0.214

0.328

0.413

0.680

Biswas et al. (2005)

45

0.563

0.186

0.035

0.198

0.928

3.026

0.002

Chen (2009)

135

0.161

0.172

0.030

−0.177

0.499

0.933

0.351

Chen et al. (2014)

126

0.603

0.256

0.065

0.102

1.104

2.358

0.018

Chen et al. (2014)

64

0.528

0.104

0.011

0.324

0.731

5.082

0.000

Crippen and Earl (2007)

66

−0.032

0.220

0.049

−0.464

0.400

−0.145

0.884

Duffy and Azevedo (2015)

83

0.015

0.127

0.016

−0.234

0.264

0.119

0.905

Delen et al. (2014)

80

0.754

0.286

0.082

0.194

1.314

2.639

0.008

El Saadawi et al. (2010)

23

0.270

0.419

0.176

−0.552

1.092

0.644

0.520

Kauffman et al. (2011)

119

0.100

0.159

0.025

−0.211

0.411

0.632

0.527

Kim and Pedersen (2011)

170

0.415

0.111

0.012

0.197

0.634

3.731

0.000

Kondo et al. (2012)

88

−0.071

0.151

0.023

−0.366

0.225

−0.467

0.640

Kramarski and Mizrachi (2006)

43

0.242

0.177

0.031

−0.106

0.589

1.364

0.173

Ifenthaler (2012)

98

0.354

0.183

0.034

−0.005

0.714

1.933

0.053

Lee et al. (2010)

223

0.385

0.083

0.007

0.223

0.548

4.648

0.000

Lehmann et al. (2014)

131

1.200

0.230

0.053

0.749

1.652

5.211

0.000

Manlove et al. (2007)

70

−0.196

0.140

0.020

−0.470

0.077

−1.406

0.160

Marée et al. (2013)

93

0.371

0.172

0.030

0.033

0.708

2.152

0.031

Molenaar et al. (2011)

156

0.055

0.071

0.005

−0.084

0.194

0.773

0.440

Molenaar et al. (2012)

110

0.124

0.191

0.036

−0.251

0.498

0.647

0.517

Moos and Azevedo (2008)

43

−0.016

0.329

0.108

−0.661

0.629

−0.049

0.961

Nückles et al. (2009)

103

0.876

0.114

0.013

0.652

1.099

7.684

0.000

Stahl and Bromme (2009)

51

0.401

0.228

0.052

−0.045

0.848

1.763

0.078

van den Boom et al. (2004)

42

−0.105

0.437

0.191

−0.962

0.752

−0.241

0.810

Wang (2011)

123

3.729

0.211

0.045

3.315

4.143

17.671

0.000

Zhang et al. (2015)

51

1.699

0.231

0.053

1.246

2.151

7.352

0.000

Characteristics of self-regulated learning scaffolds

The characteristics of scaffolds were analyzed in terms of mechanism, functions, delivery forms, mode, numbers of scaffolds, and how to facilitate SRL by scaffolds.

The results by the mechanism of scaffolds are shown in Table 4. Twenty-one studies (72.4 %) adopted prompts or hints to scaffold SRL. Six studies (20.8 %) employed an integrated SRL tool as the mechanism of scaffolding, which was associated with the highest effect size. One study (3.4 %) used the concept map as the mechanism; another study (3.4 %) adopted the worked example as its mechanism, which was correlated with the lowest and negative effect size. Furthermore, there was no significant between-group effect among the four kinds of mechanism (Q = 4.139, df = 3, p = 0.247).
Table 4

Analysis results by mechanisms of scaffolds

 

Sample size

ES

SE

Variance

95 % CI

Z

p

Test of heterogeneity in effect size

Lower

Upper

Q

df

p

Prompts or hints

1813

0.317

0.081

0.007

0.157

0.477

3.891

0.000

   

Concept map

93

0.371

0.172

0.030

0.033

0.708

2.152

0.031

   

Worked-out example

66

−0.032

0.220

0.049

−0.464

0.400

−0.145

0.884

   

Integrated SRL tool

616

0.947

0.502

0.252

−0.037

1.930

1.887

0.059

   

Total between (Q B)

        

4.139

3.0

0.247

With regard to the functions of scaffolds, 19 studies (65.5 %) adopted multiple functions when designing the scaffolds. Four studies (13.8 %) preferred a metacognitive focus and another four (13.8 %) designed strategic scaffolds to support SRL. Two studies (6.9 %) adopted conceptual scaffolds. The highest effect size was associated with the strategic scaffolds, and the lowest effect size was correlated with the conceptual scaffolds (Table 5). However, there was no significant difference among the four categories (Q = 5.518, df = 3, p = 0.138).
Table 5

Analysis results by functions of scaffolds

 

Sample size

ES

SE

Variance

95 % CI

Z

p

Test of heterogeneity in effect size

Lower

Upper

Q

df

p

Conceptual

154

0.064

0.110

0.012

−0.151

0.279

0.586

0.558

   

Strategic

335

1.041

0.958

0.919

−0.837

2.920

1.087

0.227

   

Metacognitive

306

0.256

0.096

0.009

0.068

0.444

2.664

0.008

   

Multiple

1853

0.376

0.090

0.008

0.200

0.552

1.087

0.277

   

Total between (Q B)

        

5.518

3

0.138

With respect to the delivery form of scaffolds, 27 studies (93.2 %) used an indirect delivery form and one (3.4 %) used direct delivery. Only one study (3.4 %) adopted both direct and indirect delivery forms. The effect size for the direct delivery form (ES = 0.354) was lower than for indirect delivery (ES = 0.414). The highest effect size was associated with both direct and indirect delivery forms. There was significant difference among these three kinds of delivery forms (Q = 10.43, df = 2, p = 0.005) (Table 6).
Table 6

Analysis results by delivery forms of scaffolds

 

Sample size

ES

SE

Variance

95 % CI

Z

p

Test of heterogeneity in effect size

Lower

Upper

Q

df

p

Direct

98

0.354

0.183

0.034

−0.005

0.714

1.933

0.053

   

Indirect

2419

0.414

0.112

0.013

0.195

0.633

3.701

0.000

   

Both direct and indirect

131

1.200

0.230

0.053

0.749

1.652

5.211

0.000

   

Total between (Q B)

        

10.43

2

0.005

In terms of mode of scaffolds, three studies (10.3 %) were coded as domain-specific scaffolds and 25 studies (86.2 %) were coded as domain-general. One study (3.4 %) was coded as using both modes. The highest effect size was associated with both modes (ES = 1.699). The lowest effect size was correlated with the domain-specific scaffolds (ES = 0.102) (Table 7). The test of heterogeneity in effect sizes indicated a significant difference between the two kinds of modes (Q = 42.77, df = 2, p = 0.000). This implies that both the domain-general and domain-specific scaffolds had a positive effect on academic performance.
Table 7

Analysis results by delivery modes of scaffolds

 

Sample size

ES

SE

Variance

95 % CI

Z

p

Test of heterogeneity in effect size

Lower

Upper

Q

df

p

Domain-general scaffolds

2392

0.432

0.117

0.014

0.203

0.661

3.693

0.000

   

Domain-specific scaffolds

205

0.102

0.086

0.007

−0.067

0.272

1.185

0.236

   

Both

51

1.699

0.231

0.053

1.246

2.151

7.352

0.000

   

Total between (Q B)

        

42.77

2

0.000

Of 29 articles, 18 (62.01 %) were coded as having multiple scaffolds and 11 (37.93 %) were coded as a unique scaffold. The higher effect size was associated with the multiple kinds of scaffolds (ES = 0.577), in contrast to the unique scaffold (ES = 0.211). There was a statistically significant difference between the multiple and unique scaffolds (Q = 4.424, df = 1, p = 0.035) (Table 8). This result showed that the multiple scaffolds were more effective in improving academic performance than the unique one. This result was also corroborated by Lee et al. (2010) who provide empirical evidence that the combination of generative learning strategy prompts with metacognitive feedback can improve learners’ comprehension of science topics.
Table 8

Analysis results by numbers of scaffolds

 

Sample size

ES

SE

Variance

95 % CI

Z

p

Test of heterogeneity in effect size

Lower

Upper

Q

df

p

Unique

779

0.211

0.079

0.006

0.056

0.366

2.662

0.008

   

Multiple

1869

0.577

0.155

0.024

0.273

0.882

3.714

0.000

   

Total between (Q B)

        

4.424

1

0.035

As to how to facilitate SRL by scaffolds, three types were coded for the 29 articles. Twenty-six studies (89.8 %) were coded as supporting the whole process of SRL. The remaining three studies supported monitoring, adaptation, and reflection specifically. Supporting the whole process of SRL was associated with the highest effect size (ES = 0.469) and supporting reflection with the lowest effect size (ES = −0.105). No statistically significant difference was found among the three categories (Q = 4.561, df = 3, p = 0.207) (Table 9). This result demonstrated that supporting the whole process of SRL by scaffolds was more helpful for improving academic performance than only supporting one specific phase of SRL.
Table 9

Analysis results by how to facilitate SRL by scaffolds

 

Sample size

ES

SE

Variance

95 % CI

Z

p

Test of heterogeneity in effect size

Lower

Upper

Q

df

p

Support monitoring

93

0.371

0.172

0.030

0.033

0.708

2.152

0.031

   

Support reflection

42

−0.105

0.437

0.191

−0.962

0.752

−0.241

0.810

   

Support adaptation

119

0.100

0.159

0.025

−0.211

0.411

0.632

0.527

   

Support the whole process of SRL

2349

0.469

0.117

0.014

0.240

0.697

4.023

0.000

   

Total between (Q B)

        

4.561

3

0.207

Demographics of the studies

Five key demographics were examined: sample groups, sample size, learning domain, research settings, and types of computer-based learning environments. For research sample groups, three main kinds of groups were selected across the 29 articles, namely primary school (four studies), junior and senior high school (eight studies), and higher education (17 studies). The higher education group was most often targeted (58.6 %), and the primary school sample was least selected (13.8 %). This may be because it is easier for researchers working in higher education to access students engaged in higher education. The smallest effect size was associated with the primary school group (ES = 0.277), while the largest was for the junior and senior high school group (ES = 0.841). There was no significant difference between the three sample groups (Q = 2.154, df = 2, p = 0.341) (Table 10).
Table 10

Analysis results by research sample groups

 

Sample size

ES

SE

Variance

95 % CI

Z

p

Test of heterogeneity in effect size

Lower

Upper

Q

df

p

Primary school

506

0.277

0.124

0.015

0.033

0.520

2.228

0.026

   

Junior and senior high school

698

0.841

0.374

0.140

0.109

1.574

2.252

0.024

   

Higher education

1444

0.287

0.086

0.007

0.120

0.455

3.355

0.001

   

Total between (Q B)

        

2.154

2

0.341

Slavin and Smith (2009) indicated that sample size may result in variation. Studies with small sample sizes were likely to yield larger effect sizes (Cheung and Slavin 2013). Therefore, the present study also examined the different effect sizes linked to different sample size. Ten studies (34.5 %) selected the larger sample size (101–300 participants) for greater statistical power. Thirteen studies (44.8 %) selected a medium sample size (50–100 participants), and six studies (20.7 %) selected the smaller sample size (20–50 participants). As shown in Table 11, the highest effect size was correlated with the largest sample size. No statistically significant difference among different sample sizes was found in the current study (Q = 2.709, df = 2, p = 0.258).
Table 11

Analysis results by sample size

 

Sample size

ES

SE

Variance

95 % CI

Z

p

Test of heterogeneity in effect size

Lower

Upper

Q

df

p

20–50

247

0.252

0.087

0.008

0.081

0.423

2.890

0.004

   

51–100

1025

0.369

0.125

0.016

0.123

0.614

2.941

0.003

   

101–300

1376

0.638

0.228

0.052

0.191

1.086

2.798

0.005

   

Total between (Q B)

        

2.709

2

0.258

In terms of research learning domain, the natural science learning domain was found in 18 studies (62.1 %) and the social science domain featured in 10 (34.5 %). Only one study was set in the medical science learning domain. As shown in Table 12, the largest mean effect size was associated with natural science (ES = 0.458) and the smallest effect size was correlated with medical science (ES = 0.270). However, there was no significant difference among the three kinds of learning domain (Q = 0.186, df = 2, p = 0.911).
Table 12

Analysis results by learning domain

 

Sample size

ES

SE

Variance

95 % CI

Z

p

Test of heterogeneity in effect size

Lower

Upper

Q

df

p

Natural science

1784

0.458

0.151

0.023

0.163

0.753

3.040

0.002

   

Social science

841

0.418

0.149

0.022

0.126

0.709

2.807

0.005

   

Medical science

23

0.270

0.419

0.176

−0.552

1.092

0.644

0.520

   

Total between (Q B)

        

0.186

2

0.911

For the research setting variable, 19 studies (65.5 %) were conducted in the laboratory and ten studies (34.5 %) were carried out in the classroom. The higher mean effect size was associated with the classroom settings (ES = 0.757), and the lower effect size was correlated with the laboratory condition (ES = 0.265) (Table 13). No significant difference between the two kinds of research settings was found (Q = 2.655, df = 1, p = 0.103).
Table 13

Analysis results by research settings

 

Sample size

ES

SE

Variance

95 % CI

Z

p

Test of heterogeneity in effect size

Lower

Upper

Q

df

p

Laboratory

1689

0.265

0.077

0.006

0.114

0.416

3.436

0.001

   

Classroom

959

0.757

0.292

0.085

0.185

1.330

2.592

0.010

   

Total between (Q B)

        

2.655

1

0.103

As to the types of computer-based learning environments, six types of environments were adopted in the 29 articles. These included hypermedia learning environment, Web-based learning environment, concept mapping tool, mobile learning environment, agent, and inquiry learning environment. The Web-based learning environment (15 studies, 51.72 %) was most used, and the mobile and inquiry learning environments were the least adopted (one study each). Six studies adopted a hypermedia learning environment: four focused on agents, and two used a concept mapping tool. The greatest effect size was associated with the Web-based learning environment (ES = 0.676) (Table 14). This suggested that scaffolding in a Web-based learning environment is more effective than for other learning environments. The mobile devices and inquiry learning environment had a negative effect size. Significant differences were found among the six categories of learning environments (Q = 19.21, df = 5, p = 0.002).
Table 14

Analysis results by types of CBEs

 

Sample size

ES

SE

Variance

95 % CI

Z

p

Test of heterogeneity in effect size

Lower

Upper

Q

df

p

Client–server-based hypermedia learning environment

458

0.225

0.158

0.025

−0.084

0.534

1.426

0.154

   

Web-based learning environment

1580

0.676

0.189

0.036

0.306

1.045

3.586

0.000

   

Concept mapping tool

191

0.363

0.126

0.016

0.117

0.609

2.892

0.004

   

Mobile learning environment

88

−0.071

0.151

0.023

−0.366

0.225

−0.467

0.640

   

Agent

261

0.218

0.141

0.020

−0.059

0.494

1.542

0.123

   

Inquiry learning environment

70

−0.196

0.140

0.020

−0.470

0.077

−1.406

0.160

   

Total between (Q B)

        

19.21

5

0.002

Research methodological features

Four key methodological features were examined: the research methods, type of research design, type of organization for treatment, and duration of treatment.

Table 15 shows 29 studies adopted the quantitative methods to analyze the data. All of the studies selected the quantitative methods because they offered descriptions of the empirical investigations of the effect of scaffolds via statistical techniques. The effect size achieved was 0.438.
Table 15

Analysis results by research methods

 

Sample size

ES

SE

Variance

95 % CI

Z

p

Test of heterogeneity in effect size

Lower

Upper

Q

df

p

Quantitative methods

2648

0.438

0.108

0.012

0.227

0.649

4.069

0.000

0.000

0

1.000

For research design, 22 studies (75.9 %) adopted a true experimental design and seven studies (24.1 %) were the quasi-experimental. Table 16 shows the effect size for the quasi-experimental design (ES = 0.486) was higher than for the true experimental design (ES = 0.419). However, the result of the heterogeneity test revealed that there was no significant difference between the true experimental design and the quasi-experimental design (Q = 0.098, df = 1, p = 0.754).
Table 16

Analysis results by research design

 

Sample size

ES

SE

Variance

95 % CI

Z

p

Test of heterogeneity in effect size

Lower

Upper

Q

df

p

True experimental design

1971

0.419

0.134

0.018

0.157

0.682

3.129

0.002

   

Quasi-experimental design

677

0.486

0.164

0.027

0.165

0.807

2.965

0.003

   

Total between (Q B)

        

0.098

1

0.754

In terms of how participants were organized for treatment, 24 studies (82.8 %) implemented scaffolds individually and five studies (17.2 %) adopted scaffolds in groups. As indicated in Table 17, a higher effect size was associated with the individual treatment (ES = 0.496) and a lower effect size was associated with groups. Moreover, a significant difference was found between the two types of organization type (Q = 7.079, df = 1, p = 0.008).
Table 17

Analysis results by types of organization for treatment

 

Sample size

ES

SE

Variance

95 % CI

Z

p

Test of heterogeneity in effect size

Lower

Upper

Q

df

p

Individual

2127

0.496

0.132

0.017

0.237

0.755

3.760

0.000

   

Group

521

0.116

0.055

0.003

0.008

0.223

2.112

0.035

   

Total between (Q B)

        

7.079

1

0.008

The duration of treatment is crucial in a meta-analysis. Table 18 presents the results regarding different durations for treatment. Seven studies (24.1 %) were coded as 2–4 weeks, associated with the highest effect size (ES = 0.776). One study (3.4 %) was coded as 9–24 weeks, associated with the lowest and negative effect size (ES = −0.032). There was no significant difference among different kinds of duration (Q = 9.83, df = 5, p = 0.080).
Table 18

Analysis results by duration of treatment

 

Sample size

ES

SE

Variance

95 % CI

Z

p

Test of heterogeneity in effect size

Lower

Upper

Q

df

p

Less than one hour

297

0.097

0.108

0.012

−0.114

0.308

0.900

0.368

   

1–10 h

1297

0.353

0.111

0.012

0.135

0.570

3.182

0.001

   

1–7 days

245

0.443

0.312

0.098

−0.169

1.056

1.420

0.156

   

2–4 weeks

698

0.776

0.355

0.126

0.081

1.472

2.187

0.029

   

5–8 weeks

45

0.563

0.186

0.035

0.198

0.928

3.026

0.002

   

9–24 weeks

66

−0.032

0.220

0.049

−0.464

0.400

−0.145

0.884

   

Total between (Q B)

        

9.83

5

0.080

Publication bias

To test for publication bias, the classic fail-safe N was conducted to examine whether there were numbers of studies with null or negative results that have not been discovered (Rothstein et al. 2006). As shown in Table 19, the results of this test revealed that 1315 studies with null results would be needed to nullify the overall effect size. Further examination of a funnel plot demonstrated that the studies were symmetrically distributed around the combined effect size. Therefore, no publication bias was found in the current meta-analysis.
Table 19

Classic fail-safe N

Items

Value

Z value for observed studies

13.34

p value for observed studies

0.00

Alpha

0.05

Tails

2.00

Z for alpha

1.96

Number of observed studies

29

Number of missing studies that would bring p value to > alpha

1315

Sensitivity analysis

Sensitivity analysis was conducted to examine outliers that may skew the overall results. The one-study removal analysis (Borenstein et al. 2011) was adopted to check for the extreme positive and negative effect sizes for the overall effect size. The results indicated that the effect sizes still fall on the 95 % confidence interval (0.229–0.648). This means that the overall effect size would not be affected if any one effect size was removed.

Discussion

The findings of this meta-analysis with a total sample size of 2648 participants indicated that SRL scaffolds in computer-based learning environments produced a medium positive effect (ES = 0.438) on academic performance. This is consistent with a previous report by Schmid et al. (2014) who found that the overall weighted average effects of technology use on achievement were significant. This result was also corroborated by a recent meta-analysis reporting a positive overall effect size for ICT interventions (Archer et al. 2014). The results of meta-analyses are more accurate and reliable than individual studies because they focus on a larger sample size (Li and Ma 2010; van Ewijk 2011). The sample size in the present study was 2648, which supports the validity and accuracy of the results.

Findings suggest that a Web-based learning environment is optimal for supporting SRL. The scaffolds should support the whole process of SRL, ranging from setting goals, making plans, and enacting strategies, to adapting metacognition. Scaffolding only one phase of SRL is not enough. This result was unsurprising because SRL is an iterative and cyclical process that involves defining tasks, setting goals and making plans, enacting tactics, and adapting metacognition (Winne 2001). Therefore, it is necessary to use scaffolds to support SRL during the whole process. Integrated SRL tools with multiple functions such as setting goals, making plans, enacting strategies, self-monitoring, and self-evaluation were found to be the most effective in promoting SRL in our study.

Similar to Ifenthaler’s (2012) report, our results also suggest that domain-general scaffolds will be more effective than domain-specific scaffolds. Since self-regulation is domain-general in nature, this is not an unexpected finding (Pintrich 2000). Scaffolds that can provide a wide variety of domain-general skills including setting goals, planning, monitoring, evaluating, and adaptation can help learners to be more self-regulated and give them greater autonomy (Koedinger and Aleven 2007). However, when used in combination, both domain-general and domain-specific scaffolds had the highest effect size. Therefore, both types of scaffolds should be designed and integrated into interventions. Additionally, the combination of different kinds of scaffolds such as prompts and feedback can achieve better academic performance. It should be noted that SRL has the contextual characteristics that include the characteristics of learners, the learning environment, and their interactions (Efklides 2011). Therefore, the design of scaffolds should also consider these contextual factors to foster the skills of SRL.

The present study indicated that there were few adaptive scaffolds used to promote SRL. Adaptive scaffolding may be more beneficial for fostering SRL and lead to significant learning gains since it adjusts based on students’ learning needs (Azevedo et al. 2004, 2005). Previous studies have provided empirical evidence to support adaptive scaffolding in different learning domains, such as algebra (Kramarski and Hirsch 2003), biology (Azevedo et al. 2004), and history (Poitras and Lajoie 2014). Adaptive scaffolds can flexibly fade based on the ongoing diagnosis and evaluation of students’ knowledge and skills so that students ultimately internalize the learning process by themselves (Azevedo and Hadwin 2005).

In addition to the overall findings, the current study investigated the moderate effect on academic performance by considering the demographics of the selected studies as well as methodological features. Key findings from the meta-analysis and the way in which these impact student academic performance are now discussed.

First, although the meta-analysis indicated that no significant difference in the effect size of different research sample groups, sample sizes, learning domains, and research settings, our findings revealed that the larger sample size (101–300) can produce a larger effect size. This result was consistent with Chang and Lin (2013) who reported that the greatest mean effect size was correlated with studies with the largest sample size (101–300). This is logical because statistical power is enhanced when a sample size increases. Therefore, it is suggested that researchers should adopt the largest sample size possible so as to obtain significant statistical power and reliable results.

Second, the study revealed that the effect size of the quasi-experimental design was higher than for the true experimental design. This finding was consistent with previous studies (Cheung and Slavin 2013). Since the results of quasi-experimental studies have larger variances (Niemiec et al. 1987), more studies with true experimental designs should be adopted in future.

Third, the findings also indicated that the effect size was the highest when the treatment continued for 2–4 weeks. If the treatment lasted too long, for example 9–24 weeks, the effect size decreased dramatically. This result was supported by Clark (1983) who believed that effect size decreased when treatment continued for a long period of time.

The current study was constrained by several limitations. First, the meta-analysis only focused on measures of academic performance. It would be very useful to assess other outcomes arising from SRL scaffolds. Future studies need to examine other outcome variables, including self-regulated skills, motivations, and emotions. Second, the study only identified three main categories of moderators, namely the scaffold characteristics, demographics, and research methodologies of the selected studies. Ongoing research is needed to investigate other potential moderators to understand how to maximize academic performance when scaffolding SRL in a computer-based environment. Finally, only empirical studies were included in the present analysis. There is much to be learned from qualitative research. Future studies will analyze qualitative research to add further to our understanding of the effects of scaffolds.

Conclusion

The overall conclusion emerging from the present meta-analysis is that SRL scaffolds in a computer-based environment have positively effects on academic performance. The characteristics of scaffolds and the demographics and methodological features of studies were examined as moderating variables. Major findings revealed that both domain-general and domain-specific scaffolds can support the entire process of SRL and positively affected academic performance. It is suggested that integrated SRL tools with multiple functions and multiple kinds of scaffolds should be considered as the optimal way to promote academic performance. In addition, large sample size and true experimental designs should be adopted to obtain optimal statistical power. To eliminate confounding variables, the duration of treatment should not be too long. The main contribution of this study lies in its exploration of moderator variables to gain insights into how scaffolding treatments and study features lead to positive effects. The study contributes to the body of knowledge about how to design scaffolds to promote SRL so as to maximize students’ learning.

Notes

Acknowledgments

This study was supported by the 2014 Beijing youth project of the Twelfth Five-Year Plan for Educational Science “The empirical study on improving the self-regulated learning abilities in smart learning environment” (CJA14185).

References

  1. Adeyinka, T., & Mutula, S. (2010). A proposed model for evaluating the success of WebCT course content management system. Computers in Human Behavior, 26(6), 1795–1805.CrossRefGoogle Scholar
  2. Aleven, V., Roll, I., McLaren, B. M., & Koedinger, K. R. (2010). Automated, unobtrusive, action-by-action assessment of self-regulation during learning with an intelligent tutoring system. Educational Psychologist, 45(4), 224–233.CrossRefGoogle Scholar
  3. Archer, K., Savage, R., Sanghera-Sidhu, S., Wood, E., Gottardo, A., & Chen, V. (2014). Examining the effectiveness of technology use in classrooms: A tertiary meta-analysis. Computers & Education, 78, 140–149.CrossRefGoogle Scholar
  4. Azevedo, R., Cromley, J. G., & Seibert, D. (2004). Does adaptive scaffolding facilitate students’ ability to regulate their learning with hypermedia? Contemporary Educational Psychology, 29(3), 344–370.CrossRefGoogle Scholar
  5. Azevedo, R., Cromley, J. G., Winters, F. I., Moos, D. C., & Greene, J. A. (2005). Adaptive human scaffolding facilitates adolescents’ self-regulated learning with hypermedia. Instructional Science, 33(5–6), 381–412.CrossRefGoogle Scholar
  6. Azevedo, R., & Hadwin, A. F. (2005). Scaffolding self-regulated learning and metacognition–Implications for the design of computer-based scaffolds. Instructional Science, 33(5), 367–379.CrossRefGoogle Scholar
  7. Azevedo, R., Moos, D. C., Johnson, A. M., & Chauncey, A. D. (2010). Measuring cognitive and metacognitive regulatory processes during hypermedia learning: Issues and challenges. Educational Psychologist, 45(4), 210–223.CrossRefGoogle Scholar
  8. Azevedo, R., Verona, M. E., & Cromley, J. G. (2001). Fostering students’ collaborative problem solving with RiverWeb. In J. D. Moore, C. L. Redfield, & W. L. Johnson (Eds.), Artificial intelligence in education: Al-ED in the wired and wireless future (pp. 167–175). Amsterdam: IOS Press.Google Scholar
  9. Azevedo, R., Witherspoon, A., Chauncey, A., Burkett, C., & Fike, A. (2009). MetaTutor: A meta cognitive tool for enhancing self-regulated learning. In R. Pirrone, R. Azevedo, & G. Biswas (Eds.), Proceedings of the AAAI fall symposium on cognitive and metacognitive educational systems (pp. 14–19). Menlo Park, CA: Association for the Advancement of Artificial Intelligence (AAAI) Press.Google Scholar
  10. Bangert-Drowns, R.L., & Rudner, L.M. (1991). Meta-analysis in educational research. Paper presented to ERIC Clearinghouse on Tests, Measurement, Evaluation, Washington, DC. (ERIC Document Reproduction Service No. ED 339748).Google Scholar
  11. Bannert, M., & Reimann, P. (2012). Supporting self-regulated hypermedia learning through prompts. Instructional Science, 40(1), 193–211.CrossRefGoogle Scholar
  12. Bannert, M., Sonnenberg, C., Mengelkamp, C., & Pieger, E. (2015). Short-and long-term effects of students’ self-directed metacognitive prompts on navigation behavior and learning performance. Computers in Human Behavior, 52, 293–306.CrossRefGoogle Scholar
  13. Benz, B. F. (2008). Self-regulated learning and academic achievement: A meta-analysis. International Journal of Psychology, 43(3–4), 541.Google Scholar
  14. Biswas, G., Leelawong, K., Schwartz, D., Vye, N., & The Teachable Agents Group at Vanderbilt. (2005). Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence, 19(3–4), 363–392.CrossRefGoogle Scholar
  15. Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2011). Introduction to meta-analysis. Chichester, UK: Wiley.Google Scholar
  16. Casal, G. B., Ávila, J. A. O., Lechuga, B. M., & Zych, I. (2011). The h index of the presidents of the American Psychological Association (APA) through journal articles included in the Web of Science database. International Journal of Clinical and Health Psychology, 11(1), 95–107.Google Scholar
  17. Chang, M. M., & Lin, M. C. (2013). Strategy-oriented web-based English instruction—A meta-analysis. Australasian Journal of Educational Technology, 29(2), 203–216.CrossRefGoogle Scholar
  18. Chen, C. M. (2009). Personalized E-learning system with self-regulated learning assisted mechanisms for promoting learning performance. Expert Systems with Applications, 36(5), 8816–8829.CrossRefGoogle Scholar
  19. Chen, C. M., & Huang, S. H. (2014). Web-based reading annotation system with an attention based self-regulated learning mechanism for promoting reading performance. British Journal of Educational Technology, 45(5), 959–980.CrossRefGoogle Scholar
  20. Chen, C. M., Wang, J. Y., & Chen, Y. C. (2014). Facilitating English-language reading performance by a digital reading annotation system with self-regulated learning mechanisms. Journal of Educational Technology & Society, 17(1), 102–114.Google Scholar
  21. Cheung, A. C., & Slavin, R. E. (2013). The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A meta-analysis. Educational Research Review, 9, 88–113.CrossRefGoogle Scholar
  22. Chung, M. K. (2000). The development of self-regulated learning. Asia Pacific Education Review, 1(1), 55–66.CrossRefGoogle Scholar
  23. Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445–459.CrossRefGoogle Scholar
  24. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46.CrossRefGoogle Scholar
  25. Cooper, H. M. (2010). Research synthesis and meta-analysis: A step-by-step approach (4th ed.). Los Angeles, CA: Sage.Google Scholar
  26. Crippen, K. J., & Earl, B. L. (2007). The impact of web-based worked examples and self-explanation on performance, problem solving, and self-efficacy. Computers & Education, 49(3), 809–821.CrossRefGoogle Scholar
  27. Dabbagh, N., & Kitsantas, A. (2005). Using web-based pedagogical tools as scaffolds for self-regulated learning. Instructional Science, 33(5–6), 513–540.CrossRefGoogle Scholar
  28. Delen, E., Liew, J., & Willson, V. (2014). Effects of interactivity and instructional scaffolding on learning: Self-regulation in online video-based environments. Computers & Education, 78, 312–320.CrossRefGoogle Scholar
  29. Devolder, A., van Braak, J., & Tondeur, J. (2012). Supporting self-regulated learning in computer-based learning environments: Systematic review of effects of scaffolding in the domain of science education. Journal of Computer Assisted learning, 28(6), 557–573.CrossRefGoogle Scholar
  30. Dignath, C., Buettner, G., & Langfeldt, H. P. (2008). How can primary school students learn self-regulated learning strategies most effectively?: A meta-analysis on self-regulation training programmes. Educational Research Review, 3(2), 101–129.CrossRefGoogle Scholar
  31. Dignath, C., & Büttner, G. (2008). Components of fostering self-regulated learning among students. A meta-analysis on intervention studies at primary and secondary school level. Metacognition and Learning, 3(3), 231–264.CrossRefGoogle Scholar
  32. Duffy, M. C., & Azevedo, R. (2015). Motivation matters: Interactions between achievement goals and agent scaffolding for self-regulated learning within an intelligent tutoring system. Computers in Human Behavior, 52, 338–348.CrossRefGoogle Scholar
  33. Efklides, A. (2011). Interactions of metacognition with motivation and affect in self-regulated learning: The MASRL model. Educational Psychologist, 46(1), 6–25.CrossRefGoogle Scholar
  34. El Saadawi, G. M., Azevedo, R., Castine, M., Payne, V., Medvedeva, O., Tseytlin, E., & Crowley, R. S. (2010). Factors affecting feeling-of-knowing in a medical intelligent tutoring system: The role of immediate feedback as a metacognitive scaffold. Advances in Health Sciences Education, 15(1), 9–30.CrossRefGoogle Scholar
  35. Hacker, D., Dunlosky, J., & Graesser, A. (2009). Handbook of metacognition in education. New York, NY: Routledge.Google Scholar
  36. Hannafin, M., Land, S. M., & Oliver, K. (1999). Open learning environments: foundations, methods, and models. In C. M. Reigeluth (Ed.), Instructional design theories and models (pp. 115–140). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  37. Hartley, K., & Bendixen, L. D. (2001). Educational research in the Internet age: Examining the role of individual characteristics. Educational researcher, 30(9), 22–26.CrossRefGoogle Scholar
  38. Hattie, J. A. (2009). Visible learning. A synthesis of over 800 meta-analyses relating to achievement. New York: Routledge.Google Scholar
  39. Hattie, J., Biggs, J., & Purdie, N. (1996). Effects of learning skills interventions on student learning: A meta-analysis. Review of educational research, 66(2), 99–136.CrossRefGoogle Scholar
  40. Hedges, L. V., & Olkin, I. (1986). Statistical methods for meta-analysis. San Diego, CA: Academic.Google Scholar
  41. Hogan, K., & Pressley, M. (1997). Scaffolding student learning: Instructional approaches and issues. Cambridge, MA: Brookline Books.Google Scholar
  42. Ifenthaler, D. (2012). Determining the effectiveness of prompts for self-regulated learning in problem-solving scenarios. Journal of Educational Technology & Society, 15(1), 38–52.Google Scholar
  43. Isaacson, R., & Fujita, F. (2012). Metacognitive knowledge monitoring and self-regulated learning. Journal of the Scholarship of Teaching and Learning, 6(1), 39–55.Google Scholar
  44. Jackson, S. L., Stratford, S. J., Krajcik, J., & Soloway, E. (1994). Making dynamic modeling accessible to precollege science students. Interactive Learning Environments, 4(3), 233–257.CrossRefGoogle Scholar
  45. Karadağ, E., Bektaş, F., Çoğaltay, N., & Yalçın, M. (2015). The effect of educational leadership on students’ achievement: A meta-analysis study. Asia Pacific Education Review, 16(1), 79–93.CrossRefGoogle Scholar
  46. Kauffman, D. F., Zhao, R., & Yang, Y. S. (2011). Effects of online note taking formats and self-monitoring prompts on learning from online text: Using technology to enhance self-regulated learning. Contemporary Educational Psychology, 36(4), 313–322.CrossRefGoogle Scholar
  47. Kim, H. J., & Pedersen, S. (2011). Advancing young adolescents’ hypothesis-development performance in a computer-supported and problem-based learning environment. Computers & Education, 57(2), 1780–1789.CrossRefGoogle Scholar
  48. Kinshuk, H. W. A., Sampson, D., & Chen, N.-S. (2013). Trends in educational technology through the lens of the highly cited articles Published in the Journal of Educational Technology and Society. Educational Technology & Society, 16(2), 3–20.Google Scholar
  49. Kistner, S., Rakoczy, K., Otto, B., Dignath-van Ewijk, C., Büttner, G., & Klieme, E. (2010). Promotion of self-regulated learning in classrooms: Investigating frequency, quality, and consequences for student performance. Metacognition and learning, 5(2), 157–171.CrossRefGoogle Scholar
  50. Koedinger, K. R., & Aleven, V. (2007). Exploring the assistance dilemma in experiments with cognitive tutors. Educational Psychology Review, 19(3), 239–264.CrossRefGoogle Scholar
  51. Kondo, M., Ishikawa, Y., Smith, C., Sakamoto, K., Shimomura, H., & Wada, N. (2012). Mobile assisted language learning in university EFL courses in Japan: Developing attitudes and skills for self-regulated learning. The Journal of EUROCALL, 24(02), 169–187.CrossRefGoogle Scholar
  52. Kramarski, B., & Hirsch, C. (2003). Using computer algebra systems in mathematical classrooms. Journal of Computer Assisted learning, 19(1), 35–45.CrossRefGoogle Scholar
  53. Kramarski, B., & Mizrachi, N. (2006). Online discussion and self-regulated learning: Effects of instructional methods on mathematical literacy. The Journal of Educational Research, 99(4), 218–231.CrossRefGoogle Scholar
  54. Kulik, J. A., & Kulik, C. L. C. (1989). The concept of meta-analysis. International Journal of educational research, 13(3), 227–340.CrossRefGoogle Scholar
  55. Lajoie, S. P., & Azevedo, R. (2006). Teaching and learning in technology-rich environments. In P. A. Alexander & P. H. Winne (Eds.), Handbook of educational psychology (2nd ed., pp. 803–821). Mahwah, NJ: Erlbaum.Google Scholar
  56. Lajoie, S. P., Guerrera, C., Munsie, S. D., & Lavigne, N. (2001). Constructing knowledge in the context of BioWorld. Instructional Science, 29(2), 155–186.CrossRefGoogle Scholar
  57. Lee, H. W., Lim, K. Y., & Grabowski, B. L. (2010). Improving self-regulation, learning strategy use, and achievement with metacognitive feedback. Educational Technology Research and Development, 58(6), 629–648.CrossRefGoogle Scholar
  58. Lehmann, T., Hähnlein, I., & Ifenthaler, D. (2014). Cognitive, metacognitive and motivational perspectives on preflection in self-regulated online learning. Computers in Human Behavior, 32, 313–323.CrossRefGoogle Scholar
  59. Li, Q., & Ma, X. (2010). A meta-analysis of the effects of computer technology on school students’ mathematics learning. Educational Psychology Review, 22(3), 215–243.CrossRefGoogle Scholar
  60. Manlove, S., Lazonder, A. W., & de Jong, T. (2007). Software scaffolds to promote regulation during scientific inquiry learning. Metacognition and Learning, 2(2–3), 141–155.CrossRefGoogle Scholar
  61. Marée, T. J., van Bruggen, J. M., & Jochems, W. M. (2013). Effective self-regulated science learning through multimedia-enriched skeleton concept maps. Research in Science & Technological Education, 31(1), 16–30.CrossRefGoogle Scholar
  62. Molenaar, I., Roda, C., van Boxtel, C., & Sleegers, P. (2012). Dynamic scaffolding of socially regulated learning in a computer-based learning environment. Computers & Education, 59(2), 515–523.CrossRefGoogle Scholar
  63. Molenaar, I., van Boxtel, C. A., & Sleegers, P. J. (2011). Metacognitive scaffolding in an innovative learning arrangement. Instructional Science, 39(6), 785–803.CrossRefGoogle Scholar
  64. Moos, D. C., & Azevedo, R. (2008). Exploring the fluctuation of motivation and use of self-regulatory processes during learning with hypermedia. Instructional Science, 36(3), 203–231.CrossRefGoogle Scholar
  65. Narciss, S., Proske, A., & Koerndle, H. (2007). Promoting self-regulated learning in web-based learning environments. Computers in Human Behavior, 23(3), 1126–1144.CrossRefGoogle Scholar
  66. Niemiec, R. P., Samson, G., Weinstein, T., & Walberg, H. J. (1987). The effects of computer based instruction in elementary schools: A quantitative synthesis. Journal of Research on Computing in Education, 20(2), 85–103.CrossRefGoogle Scholar
  67. Nückles, M., Hübner, S., & Renkl, A. (2009). Enhancing self-regulated learning by writing learning protocols. Learning and instruction, 19(3), 259–271.CrossRefGoogle Scholar
  68. Paris, S. G., & Paris, A. H. (2001). Classroom applications of research on self-regulated learning. Educational psychologist, 36(2), 89–101.CrossRefGoogle Scholar
  69. Perry, N. E., & Winne, P. H. (2006). Learning from learning kits: gStudy traces of students’ self-regulated engagements with computerized content. Educational Psychology Review, 18(3), 211–228.CrossRefGoogle Scholar
  70. Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, P. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 451–502). San Diego, CA: Academic Press.CrossRefGoogle Scholar
  71. Poitras, E. G., & Lajoie, S. P. (2014). Developing an agent-based adaptive system for scaffolding self-regulated inquiry learning in history education. Educational Technology Research and Development, 62(3), 335–366.CrossRefGoogle Scholar
  72. Pressley, M., & Hilden, K. (2006). Cognitive strategies. In D. Kuhn & R. S. Siegler (Eds.), Handbook of child psychology: Volume 2: Cognition, perception, and language (6th ed., pp. 511–556). Hoboken, NJ: Wiley.Google Scholar
  73. Rothstein, H. R., Sutton, A. J., & Borenstein, M. (Eds.). (2006). Publication bias in meta-analysis: Prevention, assessment and adjustments. UK: John Wiley & Sons.Google Scholar
  74. Saye, J. W., & Brush, T. (2002). Scaffolding critical reasoning about history and social issues in multimedia-supported learning environments. Educational Technology Research and Development, 50(3), 77–96.CrossRefGoogle Scholar
  75. Schmid, R. F., Bernard, R. M., Borokhovski, E., Tamim, R. M., Abrami, P. C., Surkes, M. A., et al. (2014). The effects of technology use in postsecondary education: A meta-analysis of classroom applications. Computers & Education, 72, 271–291.CrossRefGoogle Scholar
  76. Shapiro, A. M. (2008). Hypermedia design as learner scaffolding. Educational Technology Research and Development, 56(1), 29–44.CrossRefGoogle Scholar
  77. Simons, K. D., & Klein, J. D. (2007). The impact of scaffolding and student achievement levels in a problem-based learning environment. Instructional Science, 35(1), 41–72.CrossRefGoogle Scholar
  78. Sitzmann, T., & Ely, K. (2011). A meta-analysis of self-regulated learning in work-related training and educational attainment: What we know and where we need to go. Psychological Bulletin, 137(3), 421.CrossRefGoogle Scholar
  79. Slavin, R. E., & Smith, D. (2009). Effects of sample size on effect size in systematic reviews in education. Educational Evaluation and Policy Analysis, 31(4), 500–506.CrossRefGoogle Scholar
  80. Stahl, E., & Bromme, R. (2009). Not everybody needs help to seek help: Surprising effects of metacognitive instructions to foster help-seeking in an online-learning environment. Computers & Education, 53(4), 1020–1028.CrossRefGoogle Scholar
  81. van den Boom, G., Paas, F., Van Merrienboer, J. J., & Van Gog, T. (2004). Reflection prompts and tutor feedback in a web-based learning environment: Effects on students’ self-regulated learning competence. Computers in Human Behavior, 20(4), 551–567.CrossRefGoogle Scholar
  82. van Ewijk, C. D. (2011). Assessing students’ acquisition of self-regulated learning skills using meta-analysis. In B. J. Zimmerman & D. H. Schunk (Eds.), Handbook of self-regulation of learning and performance (pp. 376–390). Milton Park: Taylor & Francis.Google Scholar
  83. Veenman, M., Van Hout-Wolters, B., & Afflerbach, P. (2006). Metacognition and learning: Conceptual and methodological considerations. Metacognition and Learning, 1, 3–14.CrossRefGoogle Scholar
  84. Wang, T. H. (2011). Developing Web-based assessment strategies for facilitating junior high school students to perform self-regulated learning in an e-Learning environment. Computers & Education, 57(2), 1801–1812.CrossRefGoogle Scholar
  85. Wang, C.-Y., Wu, H.-K., Lee, S. W.-Y., Hwang, F.-K., Chang, H.-Y., Wu, Y.-T., et al. (2014). A review of research on technology-assisted school science laboratories. Educational Technology & Society, 17(2), 307–320.Google Scholar
  86. White, B. Y., Shimoda, T. A., & Frederiksen, J. R. (2000). Facilitating students inquiry learning and metacognitive development through modifiable software advisers. In S. P. Lajoie (Ed.), Computers as cognitive tools II: No more walls: Theory change, paradigm shifts and their influence on the use of computers for instructional purposes (pp. 97–132). Mawah, NJ: Erlbaum.Google Scholar
  87. Winne, P. H. (2001). Self-regulated learning viewed from models of information processing. In B. Zimmerman & D. Schunk (Eds.), Self-regulated learning and academic achievement: Theoretical perspectives (pp. 153–189). Mahwah, NJ: Lawrence Erlbaum Associates Inc.Google Scholar
  88. Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker & J. Dunlosky (Eds.), Metacognition in educational theory and practice. The Educational Psychology Series (pp. 277–304). Mahwah, NJ: Erlbaum.Google Scholar
  89. Winters, F. I., Greene, J. A., & Costich, C. M. (2008). Self-regulation of learning within computer-based learning environments: A critical analysis. Educational Psychology Review, 20(4), 429–444.CrossRefGoogle Scholar
  90. Zhang, W. X., Hsu, Y. S., Wang, C. Y., & Ho, Y. T. (2015). Exploring the impacts of cognitive and metacognitive prompting on students’ scientific inquiry practices within an e-learning environment. International Journal of Science Education, 37(3), 529–553.CrossRefGoogle Scholar
  91. Zimmerman, B. J. (2008). Investigating self-regulation and motivation: Historical background, methodological developments, and future prospects. American Educational Research Journal, 45(1), 166–183.CrossRefGoogle Scholar
  92. Zimmerman, B. J., & Bandura, A. (1994). Impact of self-regulatory influences on writing course attainment. American Educational Research Journal, 31(4), 845–862.CrossRefGoogle Scholar

Copyright information

© Education Research Institute, Seoul National University, Seoul, Korea 2016

Authors and Affiliations

  1. 1.School of Educational Technology, Faculty of EducationBeijing Normal UniversityBeijingChina

Personalised recommendations