The Effects of Problem-Based, Project-Based, and Case-Based Learning on Students’ Motivation: A Meta-Analysis [Eindrapport NRO-project 405-15-720]

In this meta-analysis, we examined the effects on students’ motivation of student-centered, problem-driven learning methods compared to teacher-centered/lecture-based learning. Specifically, we considered problem-based (PBL), project-based (PjBL), and case-based learning (CBL). We viewed motivation as a multifaceted construct consisting of students’ beliefs (competence and control beliefs), perceptions of task value (interest and importance), and reasons for engaging in tasks (intrinsic or extrinsic). In addition, we included students’ attitudes toward school subjects (e.g., science). We included 139 subsamples from the 132 included reports (83 PBL, 37 PjBL, and 19 CBL subsamples). Overall, PBL, PjBL, and CBL had a small to moderate, heterogeneous positive effect (d = 0.498) on motivation. Moderator analyses revealed that larger effect sizes were found for students’ beliefs, values, and attitudes compared to students’ reasons for studying. No differences were found between the three instructional methods on motivation. However, effect sizes were larger when problem-driven learning was applied in a single course (when compared to a curriculum-level approach). Larger effects were also found in some academic domains (i.e., healthcare and STEM) than in others. While the impact of problem-driven learning on motivation is generally positive, the intricate interplay of factors such as academic domain and implementation level underscores the need for a nuanced approach to leveraging these instructional methods effectively with regard to increasing student motivation.

Keywords: Motivation, attitude, problem-based learning, project-based learning, case-based learning. Introduction

Lifelong Learning
Since the beginning of this century, lifelong learning has been a central issue in European education policy (Commission of the European Communities, 2000;European Commission, 2001). In our current society, knowledge and skills change rapidly, therefore employees need to have the motivation and skills to continuously learn and develop themselves throughout their careers (Gijbels, Raemdonck, & Vervecken, 2010;Kyndt & Baert, 2013). Research has indicated that employees' motivational beliefs and willingness to learn are predictive of their actual participation in learning activities (Kyndt & Baert, 2013).
Although all people are believed to have a natural inclination toward learning, negative learning experiences or uncertainty can suppress this tendency (McCombs, 1991).
To foster students' willingness to learn, student-centered learning methods are becoming increasingly popular in education (Baeten, Kyndt, Struyven, & Dochy, 2010;Loyens & Rikers, 2017;Könings, Brand-Gruwel, & Van Merriënboer, 2005;Schmidt, Van der Molen, Te Winkel, & Wijnen, 2009). These methods were developed as a reaction to teachercentered learning, which focuses on the transmission of knowledge and meaning from the teacher to students. In contrast, in student-centered learning, students have an active role and make use of classroom practices such as observations, generating questions, discussion, and self-study.
These active, student-centered learning methods aim to promote students' motivation and attitudes toward learning (e.g., Blumenfeld et al., 1991;Hmelo-Silver, 2004;Schmidt et al., 2009). However, there is considerable debate about the effectiveness of these learning environments. On the one hand, there are several factors present in student-centered environments that have been shown to foster motivation. For example, students receive ample opportunities to exercise control over their own learning process, which is assumed to be motivating (Black & Deci, 2000). Furthermore, learning is often conducted in the context of challenging, meaningful, realistic tasks, which can lead to higher motivation as well (Blumenfeld, 1992;Norman & Schmidt, 1992). On the other hand, in these student-centered learning environments, there is a potential danger that too little guidance is offered during learning, which can lead to frustrations and uncertainties (Dahlgren & Dahlgren, 2002;Miflin, Campbell, & Price, 1999, and might negatively affect motivation. The objective of the current paper is to present the findings of a systematic review and metaanalysis into the effects of three commonly implemented student-centered learning environments on students' motivational beliefs. Specifically, problem-based learning (PBL), project-based learning (PJBL), and case-based learning (CBL) are examined.

Theoretical Framework: Motivation to Learn
Motivation is a complex and extensively researched subject (Murphy & Alexander, 2000;Pintrich, 2003;Pintrich & Schunk, 2002). Although many different conceptualizations of and theories on motivation exist, motivation to learn is often driven by two questions: "Can I do this task?" and "Why am I doing this task?" (see Pintrich, 2003). Figure 1 presents an overview of the motivational constructs incorporated in this review.

Can I do this task?
Students' answers to the question "Can I do this task?" are determined by their control beliefs. Perceived control constructs can be classified in three groups based on the relationship between the student who exerts control, the pathway through which it is exerted, and the desired (or undesired) learning outcomes (Skinner, 1996). The first group of perceived control constructs concerns students' beliefs about their personal capabilities or skills, such as self-efficacy beliefs (Skinner, 1996). Self-efficacy measures ask students to rate their confidence in their ability to perform certain tasks (e.g., school tasks) or skills (Bandura, 1997). The second group of constructs refers to students' beliefs about the factors that influence success in school, such as ability, effort, others, or chance (Skinner, 1996). Within these constructs an internal locus of control or student-related causes are contrasted against an external locus of control or non-student-related causes. Examples of internal causes can refer to specific behaviors (e.g., effort) or attributes (e.g., ability) of the student. In contrast, external causes are beyond the students' control and can be divided in those that emanate from others (e.g., task difficulty) or factors outside of human control (e.g., chance). It is believed that perceptions of internal control are more beneficial for learning than perceptions of external control (Pintrich, 2003). The final group of perceived control constructs refers to beliefs about one's own influence on success. It concerns the extent to which an individual can intentionally attain desired outcomes or prevent undesirable outcomes (Skinner, 1996). Examples of these beliefs are outcome expectations or competence expectancy beliefs in which students perceive a linkage between their doing and the outcome (Bandura, 1997;Pintrich, 2003).
Research demonstrated that if students feel self-efficacious in their ability to perform a study-related task, feel in control of their own learning, and perceive a link between their own actions and the outcome, they are more likely to exert effort and to obtain better performances (Bandura, 1997;Pintrich, 2003). Furthermore, control beliefs might play an important role in promoting lifelong learning. For example, Kyndt and Baert (2013) concluded in their review that an employee's self-efficacy is one the best predictors of actual engagement in workrelated learning activities. Therefore, if educational environments can positively foster effective control beliefs, students might be more inclined to learn now and in the future.
Why should I do this task? The question "Why should I do this task?" refers to three elements: 1) students' attitudes toward learning, 2) their perception of task value, and 3) students' reasons or goals for engaging in learning activities (Pintrich, 2003). An attitude toward an activity reflects the individual's global positive or negative evaluations of performing a particular activity (Ajzen, 1991;Armitage & Conner, 2001). It is often considered to be an affective evaluation of something (Germann, 1988). If students have a positive attitude toward learning they are more likely to engage in it. Attitudes toward learning are often measured in the context of Science, Technology, Engineering, and Mathematics (STEM) courses, to measure how students feel toward STEM-subject in school (Germann, 1988).
A related construct to attitude toward learning is task value. Task value is determined by weighing the positive and negative aspects or consequences of engaging in an activity. If students perceive the activity as important or useful for (future) plans or goals (e.g., finding a job), they are more likely to engage in it (Eccles, 1983;Eccles & Wigfield, 2002;Wigfield & Eccles, 2000). In addition, students' interest in performing the activity (now or in the future) is an important aspect of task value. In problem-based environments, interest is considered to be the driving force for learning (Schmidt, Rotgans, & Yew, 2011).
Students' goals or reasons to perform an activity determine if and why study-related tasks are performed as well (Pintrich, 2003). Most people are familiar with the classical distinction between intrinsic and extrinsic motivation. Intrinsically motivated students, study because they find the task interesting or enjoyable, whereas extrinsically motivated students, learn to obtain an external reward (e.g., recognition, praise, Ryan & Deci, 2000a). However, current theories that focus on goals or reasons for studying, present a more differentiated view on motivation to learn. Both achievement goal theory and self-determination theory are focused on students' reasons or goals for engaging in activities (Deci & Ryan, 2000;Elliot & McGregor, 2001;Ryan & Deci, 2000a). They are the most commonly cited motivational theories nowadays.
In achievement goal theory, goal orientation refers to how a person interprets and reacts to tasks (e.g., Dweck & Leggett, 1988;Elliot & McGregor, 2001). An important distinction is made between developing ability (i.e., learning/mastery goal orientation) versus demonstrating ability (i.e., performance goal orientation) during goal pursuit. This distinction was later refined by incorporating an approach/avoidance dimension, resulting in four types of goal orientation: mastery-approach, mastery-avoidance, performance-approach, and performance-avoidance goals (Elliot & McGregor, 2001). Individuals with a masteryapproach goal want to develop their competence, whereas people with a mastery-avoidance goal want to avoid the deterioration of their competence. In contrast, people with a performance-approach goal want to demonstrate their competence or outperform others, whereas individuals with a performance-avoidance goal want to avoid looking incompetent.
Self-determination theory presents a self-determination continuum that ranges from intrinsic motivation to amotivation (Deci & Ryan, 1985Ryan & Deci, 2000a, 2000b. The most important distinction is made between autonomous and controlled motivation (Deci & Ryan, 2008). Autonomously motivated students experience psychological freedom. They either perform activities out of interest (i.e., intrinsic motivation) or because they want to develop themselves or believe the task is important for obtaining personal life goals (i.e., identified motivation). In contrast, students with high scores on controlled motivation experience pressure from within, such as feelings of shame or guilt (i.e., introjected motivation) or from an external source, such as demands of others (i.e., external motivation).
Because the different types of motivation are expected to lie on a continuum, sometimes one composite score is calculated, the relative autonomy index (RAI; Grolnick & Ryan, 1987).
The RAI gives insight into the degree of autonomy or self-determination a student experiences. In addition to autonomous or controlled motivation, students can experience amotivation as well (Deci & Ryan, 2000;Ryan & Deci, 2000a, 2000b. If students are amotivated, they lack motivation to engage in the activity, for example, due to learned helplessness. Research has indicated that autonomous motivation has close links with mastery goal orientation, whereas controlled motivation has close links with performance goal orientation (Assor, Vansteenkiste, & Kaplan, 2009;Wijnia, Loyens, & Derous, 2011). Autonomous and mastery goal types of motivation are often assumed more beneficial outcomes than controlled and performance goal types of motivation (e.g., Deci & Ryan, 2000;Elliot & McGregor, 2001). In this review, scores on mastery goal orientation and autonomous motivation are grouped in the same category, as are performance goal orientation and controlled motivation (see Figure 1).

Problem-based, Project-Based, and Case-Based Learning
According to McCombs (1991), student-centered learning environments might foster students' motivational beliefs. In this review, we examine the effects of PBL, PjBL, and CBL environments on students' motivation to learn. These learning environments share similar characteristics (Dochy, Segers, Gijbels, & Van den Bossche, 2002: Loyens & Rikers, 2017. Firstly, teachers have a coaching instead of a directive role, whereas students have an active role instead of a passive one. Furthermore, emphasis is put on knowledge construction instead of reproduction and learning often takes place in small, collaborative groups. Finally, PBL, PjBL and CBL are all methods of instruction based on inquiry. While the exact operationalization might differ. All methods rely on the process of inquiry: generating questions and working towards answers to these questions (Loyens & Rikers, 2017). Due to the similarities between the learning environments, the terms problem-based, project-based, and case-based are sometimes used interchangeably. However, in this review, we argue that important differences exist between these three methods and that each environment has unique characteristics that might affect student outcomes differently.
Problem-based learning. PBL is a student-centered, collaborative learning method, in which small groups of students work together on meaningful, real-life problems under the guidance of a teacher (Barrows, 1996). Although various types of PBL curricula can be distinguished, researchers generally agree that PBL has five defining characteristics (Barrows, 1996;Hmelo-Silver, 2004;Schmidt et al., 2009). These characteristics include: 1) the use of problems as the start of the learning process, 2) collaborative learning in small groups, 3) student-centered, active learning, 4) the guiding role of teachers, and 5) ample time for selfstudy.
In PBL, the learning process always starts with a problem. The problem describes an event or phenomenon in daily life in need of explaining. An example of a problem was described by Schmidt (1983a, p. 387): "A red blood cell is put into pure water under a microscope. The red blood cell swells rapidly and eventually bursts. Another red blood cell is added to a solution of salt in water and it is observed to shrink." In groups of 5 to 12, students try to explain or solve the problem by use of their prior knowledge and common sense (Barrows, 1996;Schmidt, 1983b). Because students' prior knowledge is insufficient to understand the problem completely, students formulate learning issues (i.e., questions) for further self-study. During the self-study phase, students use these learning issues to collect and study new information (e.g., books, articles, websites). Students either conduct this selfstudy process alone or together with fellow students. After a period of self-study, students return to their groups to discuss their findings and apply the new knowledge to the problem.
In many PBL-environments, a large portion of time is allocated to the self-study process (Schmidt et al., 2009).
Project-based learning. PjBL has its origins in the project method described by Kilpatrick in 1918 and was later elaborated upon by other researchers, such as Blumenfeld et al. (1991;Pecore, 2015;Savery, 2006). In PjBL, the learning process is organized around projects (Loyens & Rikers, 2017;Thomas, 2000). Projects are complex tasks that are based on challenging questions or problems and drive students' learning activities. According to Thomas (2000), PjBL has the following defining characteristics: 1) projects are central to the curriculum, that is they are the main vehicles through which students learn new concepts, 2) projects are focused on questions that drive students to learn the central concepts and principles, 3) projects let students engage in knowledge construction, 4) projects are in a large part student-driven (i.e., no predetermined path exists), and 5) projects are realistic.
Similar to PBL, collaboration is important and teachers have a facilitating or coaching role (Loyens & Rikers, 2017). However, an important difference between PBL and PjBL is the role of the problem. In PBL the problem is used as a means to foster the learning process, whereas PjBL culminates in the creation of an end product that addresses the problem or question (Blumenfeld et al., 1991). The end product reflects students' new knowledge or attitudes concerning the issue under investigation and can be presented in different ways (e.g., a computer animation, website, or thesis).
Case-based learning. CBL or case-based instruction is often assumed to have its origins in the case studies and case method used at Harvard Law School and Harvard Business School (Merseth, 1991). However, PBL has been described as an important influence on the development of CBL as well (Williams, 2005). CBL is a student-centered, collaborative learning method, where students are presented with a case (Loyens & Rikers, 2017). Cases are often written as realistic problems or scenarios that need to be solved or explained. They can vary in length (paragraph vs. several pages). Small groups of students work together to solve the case. Teachers have a guiding role and facilitating role while students work on the case. CBL is often described as a special case of problem-based learning (Loyens & Rikers, 2017), however, there are some important differences. In contrast to PBL, in CBL, students are expected to have some prior knowledge on the case, therefore they need to prepare in advance for the case or are asked to apply the knowledge they have learned (Srinivasan, Wilkes, Stevenson, Nguyen, & Slaving, 2007;Williams, 2005). Moreover, teachers are more directive in bringing students back to the learning objectives (Srinivasan et al., 2007). Also, in contrast to PBL where students have to seek out additional data after being presented with the case, there is often little work/self-study after a CBL session (Srinivasan et al., 2007). In summary, important defining features of CBL are that students prepare the case in advance and that the main work is done during the CBL session.

Prior Reviews and Meta-Analyses
Problem-based, project-based, and case-based learning are often assumed to promote students' motivation, interest, and attitudes. In PBL, promoting students' intrinsic motivation for studying is even included as an important educational objective (Barrows, 1986;Hmelo-Silver, 2004;Norman & Schmidt, 1992). As mentioned, these student-centered environments have several elements that could potentially enhance motivation. For example, students have a certain amount control over and choice in their own learning process. Feelings of being in control and having a choice have been shown to enhance motivation (Black & Deci, 2000;Patall, Cooper, & Robinson, 2008). Furthermore, students learn in the context of meaningful, real-life, complex tasks, which could promote motivational beliefs as well (e.g., Ames, 1992: Blumenfeld, 1992Blumenfeld et al., 1991;Katz & Assor, 2007).
To date, several reviews and meta-analyses have been conducted about the effects of student-centered learning environments. However, most of these reviews have focused on the effectiveness of these learning environments on learning outcomes, approaches to learning, or self-directed learning skills (e.g., Baeten et al., 2010;Dochy, Segers, Van den Bossche, & Gijbels, 2003;Loyens, Magda, & Rikers, 2008). With respect to students' motivation only one prior meta-analysis was performed.
Yukhymenko (2011) conducted a meta-analysis regarding the effects of a web-based PBL program, GlobalEd Project, on students' interest in social studies and negotiation selfefficacy. The GlobalEd Project is a 4-week intervention for middle and high school students in which they learn to conduct international negotiations in the context of their social studies classroom. Several classes and schools participate in the intervention simultaneously. Each class is assigned to a specific country. During the intervention students learn about the culture, economy, history, and political structure of the participating countries.
Communications between the countries (i.e., different classes) takes place though email and online chats. The meta-analysis covered 13 applications of the GlobalEd Project that were conducted between 2002 and 2008. Results revealed a small statistically significant increase in middle school (d = 0.05, 95% CI [0.00, 0.10]) and high school (d = 0.18, 95% CI [0.12, 0.25]) students' interest in social studies. With respect to negotiation self-efficacy, only a small significant increase between pre-and posttest was found for the high school students (d = 0.08, 95% [0.02, 0.14]), whereas there was no significant increase for the middle school students (d = -0.01, 95% [-0.06, 0.04]). This meta-analysis provides some support for the assumption that environments, such as PBL, can be motivating. However, because the metaanalysis was only performed on one program, more research is needed to examine whether the results of this meta-analysis hold for other programs.
Furthermore, two meta-analyses have been conducted about the effects of PBL on students' attitudes (Batdı, 2014;Demirel & Dağyar, 2016). Batdı (2014) included 25 samples in which pre-and post-attitude measures were filled out in a PBL and a control group. A medium-sized positive effect was found in favor of PBL. Demirel and Dağyar (2016) included 47 studies that compared PBL versus "traditional" learning. They found a small, positive effect in favor of PBL as well (Hedges's g = 0.44, 95% CI [0.28, 0.60]). However, "attitude" is a broad construct (Germann, 1988). Both meta-analyses above did not exclusively focus on positive attitudes toward learning, but also included studies in which attitudes reflected a students' feelings toward the methods of teaching that were applied in a course or attitudes about issues other than learning (Batdı, 2014;Demirel & Dağyar, 2016).
However, in the current systematic review and meta-analysis, we are only interested in attitude toward learning (i.e., in general or toward a specific subject).
In conclusion, the results of previous meta-analyses suggest that PBL environments can positively affect students' interest, self-efficacy beliefs, and attitudes. However, less is known about the effectiveness of PjBL and CBL. Furthermore, the prior meta-analyses were only focused on a certain program or a specific type of design (e.g., independent groups designs).

Objectives and Research Questions
With the present systematic review and meta-analysis we aim to contribute to the knowledge base in several ways. Firstly, in addition to examining the effects of PBL, we examine the effects of project-based and case-based learning as well. Some researchers use these terms interchangeably, whereas others argue these learning environments are qualitatively different (Loyens & Rikers, 2017). By examining all three learning environments in the same metaanalysis, we can gain more insight into their possible differential effectiveness. Secondly, we examine the effects on multiple motivational outcomes. As discussed above, motivation is a multifaceted construct (e.g., Pintrich, 2003). It is possible that student-centered learning environments are more effective in fostering some motivational beliefs, while others are unaffected.
In the meta-analysis, we include studies that measure motivation on a continuous scale/Likert scale with different types of designs, such as single-group pre-post designs, independent groups posttest-only, and independent groups pre-posttest designs in which the student-centered group was compared to traditional or conventional teacher-centered instruction. By doing so, we can examine the effect of design as well. Studies that examine the effects of PBL, PjBL, and CBL on motivation longitudinally will be discussed separately.
Our research objectives were investigated by performing a systematic review and meta-analysis of the literature between 1970 and January 2017. In the review, we included gray literature, such as conference papers and dissertations as well as peer-reviewed articles.
Although, PjBL and CBL have roots before 1970, their development was partly influenced by the popularity of PBL as well (e.g., Williams, 2005). Therefore, we decided to only include studies after PBL came into existence (Neufeld & Barrows, 1974;Spaulding, 1969).
Specifically, the following main research questions are addressed in the meta-analysis: 1) What is the effect of problem-based, project-based, and case-based learning on students' motivation?
2) Do PBL, PjBL, and CBL environments have a different effect on students' motivation?
Furthermore, we examine whether systematic variance in effect sizes exists and whether this variance can be explained by several moderators, such as characteristics of the learning environment (e.g., quality of the environment, duration of the intervention, school level), study characteristics (e.g., design and publication status), and the quality of the motivational measure (for more details see "Coding Procedure").
3) Which characteristics of study, learning environment, study, or motivational measure moderate the effects of PBL, PjBL, and CBL on motivation?
Because we view motivation as a multifaceted construct, we additionally investigated the effect of PBL, PjBL, and CBL on control beliefs, attitudes toward learning, perceptions of task value, and students' goals and reasons for studying. Specifically, we are interested in answering the following research question: 4) Do PBL, PjBL, and CBL environments have different effects on students' control beliefs, attitudes toward learning, task value, and goals and reasons for studying? For motivation, several variations of the terms included in Figure 1 were used. The searches conducted in the ProQuest and Web of Science interfaces with the learning environment keywords for inquiry-based, problem-based, and project-based learning were restricted with NOT "case study", because we were only interested in quantitative research design instead of qualitative case studies. A full overview of the combination of search terms can be found in Appendix A. The search resulted in 3,554 titles of which 2,722 remained after removal of duplicates. The first and third author scanned all titles, abstracts, and, when necessary, the full texts for inclusion (95.90% interrater agreement). Disagreements (about 112 manuscripts) were resolved through discussion, and when needed, discussed with the second author. Eventually 164 of the 2,722 titles were selected for further inspection.
A key-journal in the field, Interdisciplinary Journal of Problem-Based Learning, was hand-searched and members of the Special Interest Group of the American Educational Research Association were contacted for additional papers. However, no new papers were identified through these searches. Additionally, the reference lists of selected and other relevant papers were checked for publications we had not identified in the database search, which resulted in 17 new papers. An additional, 8 papers were found by contacting authors or by checking relevant websites of authors who were included in the meta-analysis 2 . One paper was included based on the first author's personal archive.

Inclusion of Studies
The studies had to meet several criteria to be eligible for inclusion: 1) The study had to examine the effect of PBL, PjBL, or CBL on students' self-reported motivation.
3) The description of the learning environment had to meet the defining characteristics of PBL, PJBL, or CBL (as previously described).
4) The study was conducted in a school or classroom setting; studies conducted in laboratory settings or at summer camps were excluded (e.g., Schmidt, 1983a).

5)
The study had to investigate a student sample; studies conducted in professional development courses or with service teachers, residents, or patient samples were excluded.
6) The study had to be reported in English, Dutch, or German.
7) The study provided sufficient statistical data to compute effect sizes.
Additionally, we decided to exclude studies on the web-based PBL program, the GlobalEd project (see Yukhymenko, 2011) and only include studies on its successor, the GlobalEd 2 project (see Brown, Lawless, & Boyer, 2013). The GobalEd/Ed 2 program is an oftenoccurring program. We believed it was better to only focus on the effects of the "new" program instead of the older version of the program. were selected for further inspection. One study had to be excluded, because a full-text could not be retrieved. An additional, 28 studies were excluded because they did not meet the inclusion criteria. Most of these studies were excluded because they did not meet the defining characteristics of PBL, PjBL, or CBL (10 studies), used an inadequate motivation measure (6 studies), or both (1 study). Three studies on the GlobalEd Project were excluded. Some studies did not have a suitable research design (6 studies) or were not conducted in a classroom setting (2 studies). Furthermore, 14 studies were excluded because they contained duplicate data (e.g., a dissertation or conference paper that was later published).
A total of 147 articles met our inclusion criteria. When these studies did not provide sufficient statistical data to compute effect sizes, the main author was contacted with a request to provide the missing data. For 13 studies, we were unable to retrieve sufficient data to compute effect sizes; these studies were discarded. Eventually, 134 papers were selected for inclusion in the systematic review. Of these papers, 83 investigated a PBL environment, 43 a PjBL environment, and 9 a CBL environment. Because 3 of the PBL studies had a longitudinal design, these studies were discussed separately. Therefore, 131 studies were eventually included in the meta-analysis.

Figure 2.
Flow chart of the search and study process.

Coding Procedure
All included studies were coded according to a data extraction form. A separate form was developed for each of the three learning environments (see Appendix B). The data extraction form allowed for the coding of bibliographical information of the research report, the research objective, study and sample characteristics of the treatment and control group, treatment (i.e., PBL, PjBL, CBL) characteristics, and the outcome measures.
For the bibliographical information of the research report, we coded the author name(s), publication year, type of report (i.e., journal article, dissertation, conference paper, other), the journal in which the article was published, the conference at which the paper was presented, or the university at which the dissertation was defended. We additionally wrote down the exact research objectives or questions that were mentioned in the paper. Study characteristics that were coded included the country in which the study was conducted, the domain under investigation (e.g., STEM, health/medical education), research design, and the level of randomization between treatment and control group (if a control group was present).
Samples were coded according to school level (i.e., K-12 or higher education).
For each learning environment, we wrote down the definition and/or implementation of the PBL, PjBL, or CBL environment under study. This definition was checked against our own definition and used to give an indication of the quality of the learning environment. The quality of the learning environment refers to the level of detail in which the PBL, PjBL, or CBL environment/implementation was described and to what extent the defining features of the learning method were present (categories: low, moderate, high). We also coded the duration of students' exposure to the learning method. Based on Slavin (2008), a distinction was made between studies that were less than 12 weeks in duration or at least 12 weeks.
Furthermore, we determined whether only part of the study program was problem-, project-, or case-based or whether the entire curriculum was designed that way.
In line with Figure 1, the motivational outcome measures were grouped in five main categories: 1) control beliefs, 2) task value, 3) attitude, 4) goal and reasons, and 5) a general motivation constructs (i.e., a composite motivation score that includes two or more of the aforementioned categories). The category "control beliefs" consisted of three subcategories: 1.1) perceptions of ability (e.g., self-efficacy), 1.2) locus of control/attribution, and 1.3) outcome expectancy. Based on the literature, we decided to further subcategorize the "perceptions of ability" category in general perceptions of ability, perceptions about academic ability, perceptions about professional skills (e.g., teaching self-efficacy for pre-service teachers), perceptions of inquiry skills (e.g., problem-solving skills, self-study skills), and perceptions of technology/computer skills.
The "task value" category was further divided in three subcategories: 2.1) interest, 2.2) interest in future learning tasks, and 2.3) other task-value beliefs. Finally, the "goal and reasons" category was further subdivided in 4.1) autonomous or mastery, 4.2) controlled and performance, 4.3) intrinsic vs. extrinsic (i.e., high scores indicate intrinsic motivation, low scores indicate extrinsic motivation) and 4.4) amotivation.
For most motivation constructs an increase in scores or a higher mean/increase for the treatment when compared to the control group indicated a positive effect (unless otherwise stated). However, for constructs in the categories "controlled and performance" and "amotivation", a decrease/lower score was viewed as a positive. For each outcome measure we further coded information about the specificity of the construct (i.e., situation-specific or general measure), reliability, and the quality of the motivation measure. The quality of the motivation construct indicates to what extent the definition and items used were clearly described and whether these items were an accurate reflection of the motivation construct under study (categories: low, moderate, high).
Interrater agreement. All studies were coded by the first author. The quality of the learning environment was additionally coded for all studies by either the second (PBL and PjBL) and third author (CBL) so that interrater reliability could be calculated. Intraclass correlation coefficients (ICC) were used as an indication of interrater reliability (see Landers, 2015). Overall high agreement was found when comparing the quality ratings of the learning environment, this resulted in a ICC(3) of .896 for PBL, .929 for PjBL, and 1.00 for CBL.
Disagreements were resolved through discussion. The quality of all motivation constructs was coded by the first and third author. The interrater reliability analysis for the quality of the motivation constructs resulted in an ICC(3) = .867. Disagreements were again resolved through discussion.
Furthermore, parts of the coding scheme relating to the country in which the study was conducted, domain under investigation, school level, level of implementation (curriculum implementation or other), and duration of the intervention were additionally coded for 70 independent study samples by three research assistants. Interrater agreement in coding ranged from 74.29% to 98.57%. When differences were found between the first author and one of the research interns, these were discussed with the research intern and/or the third author.
Afterward, the original coding of the first author was checked and changes were made when necessary.

Data Analyses
All analyses were performed in Comprehensive Meta-Analysis statistical software (version 2; Biostat, Englewood, NJ, Borenstein, Hedges, Higgins, & Rothstein, 2009) by the fourth author. The 131 selected reports reported outcomes for 151 independent subsamples. These subsamples were used as the unit of analysis. To analyze the overall effect of the three inquiry instructional methods on student motivation, we first computed one weighted effect size for motivation using the standardized mean difference (Cohen's d). If a study included several motivational measures, these data were combined into a single effect size as an indication of overall motivation. To be able to analyze the effects for the different motivational subcategories (see Figure 1 and coding procedure), additionally one effect size was computed per category and sample. Effect sizes of .20 will be interpreted as a small effect, .50 as a medium effect, and .80 as a large effect.
If reported, we used sample size and the pretest and posttest means and standard deviations (SDs) to compute effect sizes. If the correlation between pre-and posttest data was not reported, we assumed a correlation of .50 to be able to compute the variance. For independent groups pre-posttest designs we used the posttest SD to standardize the effect size.
If no sample means or SDs were reported, we used statistical data such as t-values, F-values, or p-values, combined with sample size to compute effect sizes.
The mean effect sizes were estimated using random effects models for all outcomes, in which heterogeneity across studies was taken into account. In random effect models the mean effect size is weighted by the variance of the sample as well as the variance between studies to account for differences in sampling error related to sample size. Moderator analyses for the categorical variables were conducted based on analysis of variance (ANOVA). Between group differences in the categorical random effects analyses were tested with the Q-statistic for between group means. Results of the Meta-Analysis

Overall Effects on Motivation
The 151 Table 1. Design, quality of the learning environment, and duration of the intervention did not moderate the combined effect of PBL, PjBL, and CBL on motivation.
However, school level did moderate the effects of the three inquiry-instructional methods on motivation. A higher effect size was found for the subsamples that were examined in a higher education setting than for the subsamples/studies conducted in K-12 education. Also, the quality of the motivational measure moderated the effect. A lower effect size was found for motivation constructs that were rated high in quality relative to moderate/low quality ratings.
Furthermore, publication status moderated the effects of the inquiry-instructional methods on motivation. Higher effect sizes were found for the results published in journal articles, when compared to other reports (e.g., dissertations, conference papers). Because most of the studies were conducted in a PBL (k = 90) or PjBL environment (k = 50) the moderator analyses of Research Question 3 were conducted separately for the effects of PBL and PjBL on students' overall motivation as well. Moderator analyses problem-based learning. Effects of the moderators, such as design, quality of the learning environment, duration of exposure to PBL, school level, quality of the motivation construct, and publication status are reported in Table 2. Only one moderator, school level, had a significant effect. Again, a higher effect size was found for studies conducted in higher education than for the studies conducted in K-12 education. In addition, a trend was found for the quality of the motivation construct, studies using motivational constructs that were rated to be low in quality resulted in a larger effect size, than the studies using motivation constructs that were rated to be high in quality.

Moderator analyses project based learning.
Results of the moderator analyses for the effect of PjBL on students' motivation are reported in Table 3. For school level, quality of the motivation construct, and publication status similar effects were found when PjBL was investigated separately as when all learning environments were combined. Again, a smaller effect size was found for K-12 studies in comparison to studies conducted in higher education. Also, a lower effect size was found for studies that used motivational constructs that were rated to be high in quality, when compared to constructs that were rated to be moderate or low in quality. A larger weighted average effect size was found for journal articles, when compared to dissertations/conference papers.
Moreover, an effect was found for the duration of students' exposure to PjBL. A larger effect size was found for studies in which students' experience with PjBL was shorter than 12 weeks, than for studies in which students had more experience with PjBL. Although not statistically significant, in the 4 subsamples for which the PjBL-implementation was rated to be high in quality a larger effect on motivation was found when compared to the subsamples with a low or moderate-quality PjBL-implementation. Again, no effect was found for the design of the study (i.e., independent groups, independent groups pre-post, and single group pre-post).

Effects on Control Beliefs, Attitude, Task Value, and Goals and Reasons
Next, we examined the effects of PBL, PjBL, and CBL on students' perceptions of control beliefs, attitude, and task value, and their goals and reasons for studying to answer Research Question 4. For this analysis, we only included the subsamples for which one or more of the motivational measures could be classified in these four categories. We, therefore, excluded "general motivation" measures. We also excluded "amotivation" from the analysis as it was only measured in one subsample. We further examined differences among the three learning methods on attitudes toward learning, task value, and goals and reasons for studying. However, we have to be cautious about the effects that are found, because these categories were examined in less subsamples than control beliefs. A significant difference among the three learning environments was found for attitude toward learning, Q (2)  Finally, students' goals and reasons for studying were scrutinized. As mentioned, increases or high scores on "autonomous & mastery" constructs were indicated as positive effects, whereas increases or high scores on "controlled & performance" constructs, were coded as negative effects. A significant difference was found, Q(2) = 7.08, p = .029. Only a positive effect was found for PjBL (k = 13, d = 0.238, SE = .10). For PBL (k = 18, d = -0.003, SE = .16) and CBL (k = 2, d = -0.051, SE = .06) near-zero, negative effects were found.

Chapter 4: Studies with Longitudinal Designs
Three studies examined the effects of PBL on motivational variables longitudinally ( change from the school setting to the work setting might explain these results, as the transition from study to work is often described as stressful (Duchscher, 2009).
Overall, the results from the longitudinal studies seem to suggest that motivation to learn and feelings of internal control can increase during the first year of a PBL program.
However, because there was no comparison group, it is unclear whether similar effects would occur for students in more teacher-centered programs. Furthermore, all longitudinal studies have been conducted at the same study program and university. Therefore, more research is needed.

Aims of Meta-Analysis
Student-centered, active learning methods are becoming increasingly popular (e.g., Baeten et al., 2010). Often these learning methods are implemented, because it is assumed that they have beneficial effects on students' motivation and learning (e.g., Hmelo-Silver, 2004).
Because these learning methods often center learning around a real-life, challenging problem or project, it is believed that learning will become more meaningful and interesting (e.g., Blumenfeld, 1992). Furthermore, due to the student-centered nature of the learning environment, students can (learn to) take control of their learning process and often have a certain degree of choice in the direction the problem or project will take. The possibility of students to take autonomous control of their learning is assumed to increase students' motivation (Black & Deci, 2000). Nevertheless, not all studies have found positive effects, and some researchers have reported about motivational problems that might occur in these environments (Dolmans & Schmidt, 2006).
The main aim of this systematic review and meta-analysis was, therefore, to investigate the effect of PBL, PjBL, and CBL environments on students' motivation (Research Question 1). We additionally examined whether systematic differences existed among the three learning methods on students' motivation (Research Question 2).
Furthermore, we determined whether several moderators affected the effectiveness of these learning methods in promoting students' motivational beliefs (Research Question 3). Finally, we examined the effects of PBL, PjBL, and CBL on different subcategories of motivation (Research Question 4).

Main Results of the Meta-Analysis
To answer Research Question 1, we first combined all learning methods and all motivational outcomes in one analysis. We found a small-to-medium effect of student-centered learning methods on students' motivation. However, the effect was heterogenous, indicating that several variables might moderate the effectiveness of these learning methods on motivation.
To answer Research Question 2, the first moderator we examined was type of learning method. Results indicated a significant difference among PBL, PjBL, and CBL in their effects on students' motivation. Although, for all learning environments positive effects were found, for PBL the effect was small, whereas PjBL and CBL had moderate effects on motivation. The fact that our analyses demonstrated that these learning environments have different effects on motivation indicates that researchers should not use the terms problembased, project-based, and case-based learning interchangeably and that they should always specify the intervention/learning method they have implemented.

Effects of Moderators
To examine Research Question 3, we further analyzed whether several characteristics of the learning environment, study, and outcome measures moderated the effectiveness of the three learning methods. An important moderator was school level. Although for both school types positive effects were found of the learning methods on motivation, the effects appeared to be smaller for K-12 education than for higher education. As mentioned, PBL and CBL have their origins in higher education (Barrows;1996;Merseth, 1991;Williams, 2005). Therefore, many of the process models that were developed for learning methods, such as PBL, to facilitate learning, are specifically focused on higher education students. Furthermore, teachers have to face many challenges if they want to implement instructional methods based on inquiry in their classroom (Ertmer & Simons, 2006) as they have to take on a new facilitating or coaching role. Moreover, it has been suggested by several researchers that younger learners need more preparation before the new instructional method can be applied (Ertmer & Simons, 2006;Simons & Klein, 2007;Torp & Sage, 1998. Many of the K-12 interventions or implementations were relatively short in duration, which leaves the possibility that students might not have always been properly prepared for engaging in the new learning method. As a consequence, the effects of these "new" learning methods on motivation might have turned out lower. Also, the quality of the motivational measure seemed to affect the magnitude of the effect size. Motivational constructs that were rated to be lower in quality (e.g., unclear description in the paper, no clear theoretical framework) resulted in higher effect sizes than constructs that were rated to be high in quality. Constructs were rated as lower in quality when the construct was only measured with one items, when the construct was not well defined, and/or when it was impossible to retrieve example items of the scale. It is possible that some of the "lower-quality" constructs were in actuality good measures, however, sometimes too little information in the reports or articles was provided to get an accurate impression of the measure. A recommendation for future research is therefore to accurately describe their measures and give a clear definition of the construct researchers are measuring, for instance by providing example items.
Publication status of the research report moderated the overall effect of the three learning methods on motivation. Publication status also moderated the effect of PjBL on motivation. Although positive effects were found regardless of publication status, a small effect size was found for conference papers/dissertations and a moderate (i.e., learning methods combined) or moderate to high (i.e., PjBL) effect size for journal articles. For PBL, publication status did not affect the outcomes; there was a small positive effect for both published and gray literature. The fact that publication status significantly moderated the effects of some of the learning methods on motivation illustrates the importance of including gray literature in the meta-analysis. Only including journal articles in a meta-analysis, might inflate the magnitude of the effect size. Slavin (2008) indicated that when synthesizing the effects of educational programs, it is important to look at the duration of the study or implementation. A 12-week criterion was suggested, so that it can be examined whether the learning method can be used over an extended time period. For the PBL and the overall analyses, duration of the program (i.e., coded as at least 12 weeks or less), did not moderate the effect on motivation. However, for PjBL larger effect sizes were found for shorter implementations. A higher effect for shorter implementations of PjBL might indicate that there is a novelty effect in play. Possibly, part of the beneficial effect of PjBL on motivation can be explained by the fact students were excited about experiencing something different than their "normal" classroom experience.
Finally, design of the study (i.e., independent groups, independent groups pre-post, or single group pre-post) and quality of the implementation or description of the learning method under study did not moderate the effects of PBL, PjBL, and CBL on motivation.

Effects on Different Motivational Constructs
We further examined potential differential effects of PBL, PjBL, and CBL on students' control beliefs, attitude toward learning, task value, and goals and reason for studying (Research Question 4). For control beliefs, such as perceptions of ability, a moderate, positive effect was found for all learning environment combined. A trend suggested that the effect of PjBL on students' control beliefs was somewhat larger (moderate effect) than the effect of PBL and CBL on students' control beliefs (small-to-medium effects). Results were similar when we only investigated the effects of the learning methods on students' perceptions of ability.
For attitudes toward learning, positive effects were found for all learning methods.
However, moderator analysis revealed a significant difference among the learning methods.
Moderate-to-high and large effect sizes were found for the PjBL and CBL studies respectively. However, the effect size in the PBL studies was small but positive. The effect of PBL on attitude in our meta-analysis concur with the effects found in previous metaanalyses (Batdı, 2014;Demirel & Dağyar, 2016).
With respect to students' perceptions of task value, a positive, moderate effect size was found for PBL, PjBL, as well as for CBL. No significant differences occurred. As mentioned, learning in the context of real-life problems and projects is often assumed to increase students' interest and their perceptions of meaningfulness (e.g., Ames, 1992: Blumenfeld, 1992Blumenfeld et al., 1991;Katz & Assor, 2007). Our results seem to indicate that PBL, PjBL, and CBL can indeed increase students' perceptions of interest, importance, and utility of the learning task.
Finally, we examined students' goals and reasons for studying. Increases or high scores on "autonomous & mastery" constructs were indicated as positive effects, whereas increases or high scores on "controlled & performance" constructs, were coded as negative effects. Moderator analysis revealed a significant difference among the learning methods.
Only a small, positive effect was found for PjBL environments, whereas near-zero, negative effects were found for PBL and CBL environments. PBL methods have the instructional goal of increasing students' intrinsic motivation for studying (Barrows, 1986;Hmelo-Silver, 2004;Norman & Schmidt, 1992). Intrinsic motivation is an important aspect of autonomous motivation (Deci & Ryan, 2000). Based on the results of this meta-analysis the claim that PBL can promote students' intrinsic motivation for studying cannot be supported. However, considering the limited number of studies examining the effects of students' goal and reasons for studying, more research is needed.
In summary, in many of the analyses conducted for Research Question 4. Larger effects were found for PjBL environments than for PBL environments. A key difference between these learning environments is the role of the problem. In PBL, the problem is used as a means to foster the learning process, whereas PjBL culminates in the creation of an end product that addresses the problem or question (Blumenfeld et al., 1991). In PBL, instructors often have a certain direction in mind when they present students with a problem, whereas in PjBL the problem and project should be largely student-driven. It is possible that students in PjBL might experience more freedom in their learning process than PBL students. More research is needed to examine this assumption. Alternatively, it is also possible that the effect size of PjBL is somewhat inflated due to novelty effects. We did not identify studies that used a curriculum-wide implementation of PjBL. That is, in all studies, projects were often included as a temporary addition to the study program, which might have affected the outcomes.

Limitations
When interpreting the results of this meta-analysis, some limitations need to be considered.
First of all, some of the analyses were conducted with a relatively small number of studies.
For example, for CBL we only identified 11 subsamples. Therefore, it was impossible to conduct moderator analyses for CBL. Although, for PBL and PjBL more subsamples were identified, sample size in some of the moderator analyses was quite low. It is therefore important to interpret the results with some caution. Nevertheless, the results of our metaanalysis are a valuable addition to the field.
For all subsamples, it was determined by two authors whether or not the implementations of PBL, PjBL, and CBL were in line with the defining characteristics of these instructional methods. However, several differences can still exist among the different implementations of the instructional methods (e.g., Maudsley, 1999), which might have affected the outcomes. Although we analyzed the effects of several moderators, variability in effect sizes might also be explained by other variables that we could not include in our metaanalysis. Another variable of interest might have been the level of preparation students received before the instructional method was implemented and the level of guidance teachers provided during the implementation. However, this information was often not reported in the research reports.
Although the categories we used in our meta-analysis were based on prior research (e.g., Pintrich, 2003), differences in the operationalization of measures, even for constructs derived from the same theory, can affect the outcomes in a meta-analysis (Hulleman, Schrager, Bodmann, & Harackiewicz, 2010). However, all motivation measures were coded and labeled in categories by the first and third author, to assure that the constructs were sufficiently similar to include in the same category.

Implications for Research and Practice
It is often assumed that student-centered learning environments can increase students' motivation. Overall, the results of our meta-analysis suggest a positive, small-to-medium effect of PjBL, PBL, and CBL on motivation. Especially, positive effects were found for students' perceptions of control beliefs, attitudes toward learning, and perceptions of task value. With respect to students' goals and reasons for studying, only a small, positive effect was found for PjBL. The latter effect is in contrast to the popular belief that PBL can increase students' intrinsic motivation for studying (e.g., Hmelo-Silver, 2004).
Analyses further revealed that positive effects were found for studies conducted in K-12 settings as well as in higher education settings. However, larger effect sizes were found for higher education implementation. This finding is important for policymakers and teachers, who are interested in the implementation of problem-based, project-based, or casebased learning in their classrooms. These results imply that students and teachers in K-12 education first need some time to adapt to the learning method and that good preparation and sufficient training is needed (Ertmer & Simons, 2006).
Results further indicate that PBL, PjBL, and CBL had differential effects on motivation. For all learning methods, generally, positive effects were found. However, the effects for PjBL were often somewhat larger than the effects of PBL. Although some researchers use the terms PBL, PjBL, and CBL interchangeably, the results of our metaanalysis suggest that is all-important that researchers clearly describe the type of instructional method they are investigating, because differences in these learning methods can affect their effectiveness.
Baeten, M., Kyndt, E., Struyven, K., & Dochy, F. (2010). Using student-centered learning environments to stimulate deep approaches to learning: Factors encouraging or discouraging their effectiveness. Educational Psychology Review, 5, 243-260.      Confidence effect size ☐ All relevant information is provided ☐ Not all data is provided G2 Explanation G3 Additional comments G4 Relevant page numbers G5 Correlation between multiple outcomes *Information regarding the computations of effect sizes was coded in a separate excel-file.