Introduction

Problem-based learning (PBL), as its name implies, is learning that is driven by problems. The PBL process typically consists of three phases, namely a problem analysis, a self-directed learning, and a reporting phase (Barrows 1988; Hmelo-Silver 2004). During problem analysis, students examine the problem together with peers, make inferences based on their prior knowledge and identify questions that need to be answered in order to understand or solve the problem. After this phase, students would engage in self-directed study to work on the learning issues previously identified. When the team reconvenes during the reporting phase, students would share their findings, refining their original ideas and hypotheses in the process. Thus, the construction of knowledge in the PBL process is a result of both collaborative learning while working with peers as well as through individual self-directed learning (Schmidt 1983).

What then is the role of the tutor in PBL? A tutor is present during the problem analysis and reporting phase to facilitate and guide students’ learning process. Tutors are expected to play active roles in the scaffolding of student learning in a PBL curriculum by providing a framework that students can use to construct knowledge on their own (De Grave et al. 1999). By probing students to think more deeply and modelling for them the kinds of questions that they should be asking themselves during problem-solving, the tutor–student relationship can be viewed as a type of cognitive apprenticeship (Hmelo-Silver and Barrows 2006; Schmidt and Moust 2000; Collins et al. 1989).

As such, the behaviors of tutors in the PBL process may be expected to influence students’ learning. Although various researchers have examined the effects of tutor-related behaviors (reviewed below), their impact on the PBL process and in students’ knowledge construction remains unclear. Hence, the objective of this study was to investigate the influence of tutor-related behaviours on students’ learning process in the different PBL phases as well as on their achievement.

Several studies have focused on the behaviours related to subject-matter expertise of the tutor (Silver and Wilkerson 1991; Schmidt et al. 1993). Findings from studies on the influence of tutors’ subject-matter knowledge remain inconclusive (Davis et al. 1992; Dolmans et al. 1996). For instance, Davis et al. (1992) found differences in the performance of students favouring tutors with specific subject-matter expertise while Dolmans et al. (1996) found that tutor expertise did not influence student achievement. However, studies focusing on the differences between tutors with subject-matter knowledge and non-subject-matter knowledge from a process perspective have provided some further insights. A study conducted by Silver and Wilkerson (1991) suggested that tutors with subject-matter expertise were more inclined to play a directive role in the tutoring process, supplied more direct answers to questions posed by students, and suggested more points for discussion. Although achievement data of students were not reported, there is the suggestion that achievement can be influenced by the subject-matter expertise of the tutor, and that this expertise is expressed in particular through directing behaviours displayed in interaction with the students. In another study by Schmidt et al. (1993), findings indicated that students guided by tutors with subject-matter expertise spent significantly more time on self-directed learning as compared to those guided by non-subject-matter experts. On the other hand, despite finding effects of subject-matter experts, Davis et al. (1992) could not identify behavioural differences in tutors with subject-matter expertise and those with lesser subject-matter knowledge.

Besides subject-matter expertise, the ability of tutors to “facilitate” the learning process is believed to be important. As PBL is student-centred rather than teacher-centred, tutors avoid dispensing information, choosing to become a coach and focusing on guiding the learning process of the students instead. Tutors are required to closely follow the discussions generated amongst the students and consider when and how they might contribute to the learning process (Wetzel 1996). This suggests the need for tutors to develop facilitative skills as they are involved in questioning, probing, suggesting and challenging ideas that are raised during discussion (Maudsley 1999). Schmidt et al. (1994), for instance, compared between faculty tutors and student tutors and the results indicated that faculty tutors used their subject-matter knowledge more extensively while student tutors were better able to identify with the difficulties students experience while dealing with the problem at hand. This difference could be attributed to what was termed as ‘cognitively congruent behaviour’ that is exhibited more significantly by student tutors (Dolmans et al. 2002). Cognitive congruence can be defined as ‘the ability to express oneself in the language of the students, using the concepts they use, and explaining things in ways easily grasped by students’ (Schmidt and Moust 1995, p. 709). Thus, the student tutors are thought to be better able to understand the nature of the problems faced by students and to respond more appropriately using prompts that are more easily understood.

The concept of cognitive congruence was studied by Schmidt and Moust (1995). These authors suggested that the necessary conditions for cognitive congruence to occur included both subject-matter expertise and “social congruence.” It was proposed that tutors who are more cognitively congruent would utilize subject-matter knowledge in a better way and be more socially congruent, which ultimately translates into higher student performance. Social congruence refers to the interpersonal qualities of the tutor such as the ability to communicate informally and empathically with students, and hence being able to create a learning environment that encourages open exchange of ideas (Schmidt and Moust 1995). Subject-matter knowledge, on the other hand, would equip tutors with the ability to follow closely and contribute effectively to the discussions generated by students (Schmidt and Moust 2000). A study conducted by Kassab et al. (2006) found that effective tutors were perceived by students as those who respected their opinions, were able to establish good communications, understand their feelings and advise them on how to learn. This indicates that possessing subject-matter knowledge alone is insufficient. Without a genuine interest in the lives and learning process of the students, the tutor would lack sensitivity to the difficulties faced by students, thus hindering their ability to guide students’ learning.

The data in the study by Schmidt and Moust (1995) was analyzed using structural equations modelling, a statistical method that allows causal hypotheses to be tested by comparing the structure of correlational data with a theoretical model. Their findings indicated that social congruence directly influenced group functioning during the problem-solving process while subject-matter expertise of tutors had a slightly direct positive impact on student achievement. Furthermore, cognitive congruence, which is the combination of subject-matter expertise and social congruence, was found to influence tutorial group functioning and this indirectly affected the level of student achievement through an increase in time spent on self-study. Hence, by using structural equations modelling, Schmidt and Moust (1995) were able to establish that a higher level of achievement can be attained through effective tutoring that requires not only the tutors’ content knowledge but an ability to interact with students on a personal level as well as to utilize language that is easily understood by students. However, how exactly do these interrelated qualities of tutors affect knowledge construction during the PBL process? Which of these tutor-related behaviours are most influential on student learning? And in which of the different learning phases within the PBL process do these behaviours most extensively influence student learning? In line with the initial findings of Schmidt and Moust (1995), we hypothesized that tutors exhibiting more cognitive congruent behaviours would influence knowledge construction and acquisition at each learning phase of the PBL process. As learning in a PBL curricular is considered to be cumulative where knowledge is built upon that which was learnt in the previous learning phase (Yew et al. 2010), students under the tutorship of such tutors should be more extensively involved in the construction of knowledge and would ultimately achieve better results at the end of the learning process. Therefore, rather than to relate tutor behaviours to the outcome of PBL, the aim of this study was to investigate the effects of tutor-related behaviours on student learning during the PBL process.

Method

Participants

The participants were second-year students from 13 randomly selected classes from the Science faculty at a polytechnic in Singapore. Data were collected from the students during the third week of Semester Two in Immunology classes in the academic year of 2008–2009. Out of 262 students, data from 223 students were used in this study while the rest were removed due to incomplete sets of results. Having completed their first year of study, students were familiar with the PBL pedagogy. In total, seven tutors participated in this study and each tutor was rated by an average of 32 students. Students and tutors gave informed consent.

Educational context

The implementation of PBL at the polytechnic is based on a rather unique “One-day-one-problem” approach where students work on one problem per day. In the classroom, students are grouped into teams of less than or equal to five and one tutor to guide the learning process. A brief description of the day’s process is described below:

  • Problem analysis phase (approximately 1 h): The problem for the day is presented to the students by the tutor. Students work in teams to identify the learning issues by utilizing their prior knowledge, assumptions and experiences. After spending some time to explore the problem on their own, the tutor will generate discussion amongst the teams and to encourage students to share their ideas and thoughts about the problem. The tutor also guides students in devising initial pathways for developing a response to the problem.

  • Self-directed learning phase (approximately 4 h): Students spend their time on individual study or helping their team members when necessary. Resources such as worksheets and suggested reading texts are commonly provided by the tutor but students are also encouraged to search and use information from the internet or textbooks. During this period, the tutor also spends approximately 20 min with each team to check on their progress and strategy aimed at understanding the problem. The tutor promotes interaction and evaluation of information found by the students during their individual study. Tutors also provide guidance in constructing new knowledge and encourage students to build on each other’s ideas.

  • Reporting phase (approximately 2 h): Students are expected to connect their findings from their individual studies and demonstrate their ability to evaluate and synthesize information. Each team shares their consolidated findings and response to the problem. Students would take turns to present portions of their team’s presentation as well as to assist one another in defending their points of view and elaborating based on questions raised by peers and the tutor. The tutor encourages critical thinking and creates opportunities for students to evaluate the information presented by their peers. Key ideas would also be clarified by the tutor if necessary.

The PBL approach adopted here is rather unique in that the entire PBL cycle is completed within 1 day. However, despite the modifications, this approach remains classified as PBL based on the ‘six core characteristics of PBL’: (1) the use of authentic problems for students to work on without prior preparation so as to achieve the required knowledge, (2) students initiate their own learning whereby students work in (3) small collaborative groups under the (4) flexible tutelage of a tutor who guides the learning process. As problems are used as the starting point for learning, (5) the number of lectures are limited and (6) students would have sufficient time for self-study (Barrows 1996; Hmelo-Silver 2004; Schmidt et al. 2009). Furthermore, learning issues are generated by students and new information is acquired through self-study rather than direct instruction from the tutor (Hmelo-Silver 2004).

Materials

Problem statement and subject matter to be mastered

The problem statement for the day was entitled “A Runny Issue” and it introduced students to concepts related to the structure and functions of antibodies. Students were to explore the biological properties of the different classes of antibodies in relation to their structure as well as antigen binding. The problem statement is presented in Appendix A.

Measurement

Measurement of tutor behaviour

Tutor behaviours were assessed by asking students to complete a questionnaire adapted from Schmidt and Moust (1995). The questionnaire consists of 10 statements and students were required to indicate how much they agreed with each statement on a five-point Likert scale ranging from ‘Not true at all’ to ‘Very true for me’. The questions were crafted with the intention of gauging three core tutor behaviours, namely, social congruence, subject expertise and cognitive congruence. Examples of the statements are ‘The tutor helped us to understand the topic’, ‘The tutor showed interest in our personal lives’ and ‘The tutor used his/her content knowledge to help us’. Social congruence was measured by three items and subject-matter expertise was measured by two while cognitive congruence was measured by five items. The questionnaire is presented in Appendix B.

The reliability of the questionnaire was determined by calculating Hancock’s coefficient H for each scale as it is a construct reliability measure for latent variable systems. The recommended cut-off value by Hancock for the coefficient H is 0.70. For this particular questionnaire, the coefficient H values ranged from 0.70 (social congruence) to 0.80 (subject-matter expertise), with an average 0.75. In addition, the validity of the questionnaire was established in Schmidt and Moust (1995).

Measurement of students’ learning process

The extent of students’ learning at each PBL phase was estimated using a concept recall test. This was designed to estimate the number of relevant concepts that students were able to recall at the end of each PBL phase: problem analysis, self-directed learning and reporting (Yew et al. 2010). The concept recall test consisted of the following instruction: “List all the keywords or terminologies that are related to antibodies that you are able to recall at this stage.” The same question was given to the students at the end of each PBL phase. Students were not allowed to discuss the question with their peers or to refer to any resources.

The assumption here is that as students engage in problem analysis, self-directed learning, group discussions, and/or peer teaching, they are in fact building semantic networks of concepts related to the problem as well as making relations between their prior knowledge and new ideas (Glaser and Bassok 1989). As learning progresses, students would master more specific terminologies to articulate the newly acquired knowledge. Hence, as these networks of knowledge in their minds expand, reorganize, and become more tightly integrated, measuring the number of relevant keywords that can be recalled at any point in time can be considered an indication of the quality and progress of students’ learning.

Measurement of students’ achievement

Students’ achievement at the end of the day was measured via the implementation of an essay test. The essay was used to estimate the depth of students’ scientific knowledge by examining their ability to describe and elaborate upon the relationship between relevant concepts learned (Alao and Guthrie 1999). It consisted of a response to the following instructions: “Describe and explain as much as you know about the structure and function of antibodies”.

Procedure

In this study, the questionnaire that was used to measure tutor behaviours was administered at the end of the reporting phase. The students were informed to answer the questions in relation to their tutor of that particular day and to reflect on their involvement with the tutor during all three learning phases. The concept recall test and essay test that were used to measure learning that takes place during the “one-day-one-problem” approach adopted by the institution were also administered on the same day but at different time points.

The concept recall test was administered immediately after each PBL phase—problem analysis, self-directed learning and reporting. The essay test was administered after students had completed the final concept recall test, which was at the end of the reporting phase. When the students were attempting the essay test, they were not allowed to refer back to what they had written for the concept recall test. The concept recall test and essay test were conducted independently of each other as they served a different purpose: The concept recall test was used as a measure of students’ learning process while the essay test was used as a measure of students’ achievement at the end of the PBL process.

No time limit was set for any of these tests. The results from the questionnaire, concept recall tests and essay test were aggregated for teams under the same tutor.

Analysis

The tutor behaviours were considered the independent variables; the learning process variables were the dependent variables. Scores for each of the core tutor behaviours, social congruence, subject-matter expertise or cognitive congruence, were computed. It is standard practice to base indicators of teacher behaviours on class averages rather than on individual level data (Marsh 1991). Hence, average scores reflecting ratings of the same tutor across different classes for the three tutor-related behaviours were used during analysis.

To examine the effects of tutor-related behaviours on the learning process, analysis of covariance (ANCOVA) was used to determine if differences were because of treatment effect or by chance. The covariate used in this study was the pre-existing grade point average (GPA) score and it equates to the average grades the students have achieved in the previous semesters of their course of study. The assumption made is that the GPA score equates to the level of prior knowledge, which may affect the results for the concept recall test and essay test. Yet, it is a measurable variable that is not affected by the experimental variables. By using ANCOVA, it is possible to reduce the error variance and provide a more accurate account of the impact made by the amount of prior knowledge on the students’ learning process and achievement as ANCOVA removes the variability of the dependent variable that can be accounted for by the covariate. The average GPA score of the 223 students who participated in this study was 2.86 (SD = 0.46).

Prior to performing ANCOVA, the data representing the tutor behaviours was divided equally into three groups for each independent variable. The purpose of categorizing the data into three groups was to rank the tutors according to their level of subject-matter expertise, cognitive congruence and social congruence. The tutors were split using the 33.4% percentile and the 66.7% percentile based on the range from the data set. The subject-matter expertise for the tutors involved in this study ranged from 3.70 to 4.29 (M = 4.01, SD = 0.22) and dividing the data into groups allowed the tutors to be ranked as having a high level, medium level or low level of subject-matter expertise. This ranking exercise was also conducted for cognitive congruence that had a range of 3.41–3.98 (M = 3.65, SD = 0.20) as well as for social congruence with a range of 2.92–4.02 (M = 3.27, SD = 0.37). For each of the independent variables, there were 2 tutors in the high and low groups and 3 tutors in the medium group.

The results of the concept recall tests were analyzed by awarding 1 point to each relevant keyword listed by the student. Total scores from the concept recall tests completed after the problem analysis, self-directed learning and reporting phases were tabulated. A repetition of a keyword within each concept recall test was only counted once.

In the case of the essay tests, the “idea unit” was used as the entity for scoring (Meyer 1985; Schiefele and Krapp 1996). Answers were segmented into idea units that are defined as a statement ending with a comma, period, or ‘and’. Each idea unit was awarded with a score of 2, 1 or 0. A score of 2 was given for a completely correct idea unit, 1 for a partially correct idea unit and 0 for a completely incorrect idea unit. Inter-rater correlation between two judges for the scoring of the essay tests was r = 0.77. Differences in judgment were resolved by discussion between the judges.

Results

The means and standard deviations of the tutor-related behaviours are shown in Table 1. There were altogether seven tutors involved in this study and their level of subject-matter expertise (M = 4.01, SD = 0.22), cognitive congruence (M = 3.65, SD = 0.20) and social congruence (M = 3.27, SD = 0.37) were measured. As for the outcome measures, the scores from the concept recall tests and essay test from 223 students were used in the data analysis. The means and standard deviations of the concept recall test administered after the problem analysis phase (M = 5.63, SD = 3.35); self-directed learning phase (M = 9.64, SD = 4.08); reporting phase (M = 9.90, SD = 3.83) as well as the essay test (M = 3.57, SD = 1.94) are shown in Table 2. A correlation analysis indicated that there was a correlation between the two outcome measures ranging from 0.32 to 0.50 at a significance level of 0.01.

Table 1 Means and standard deviations of the independent variables
Full size table
Table 2 Means and standard deviations of the dependent variables
Full size table

The ANCOVA revealed that the social congruence of tutors had the most influence on the learning process relative to cognitive congruence and subject-matter expertise. Social congruence was found to have a significant effect on the total number of concepts recalled at the end of the problem analysis phase, F(2, 219) = 10.38, p < 0.01; self-directed learning phase, F(2, 219) = 9.83, p < 0.01; and reporting phase, F(2, 219) = 6.51, p < 0.01. No significant effects were found of subject-matter expertise and cognitive congruence of the tutor on each of the learning phases in the PBL process. Social congruence also had a significant effect on student achievement as measured by the essay, F(2, 219) = 4.914, p < 0.01. Similar effects were found for the subject-matter expertise, F(2, 219) = 7.74, p < 0.01, and cognitive congruence, F(2, 219) = 7.74, p < 0.01. The means and standard deviations from ANCOVA for relatively low, medium and high scoring tutors are shown in Table 3.

Table 3 Means and standard deviations of test scores with respect to tutor behaviours
Full size table

Discussion

The purpose of this study was to examine how the behaviours of tutors in a PBL curriculum would affect the students’ learning process and outcome. The results have indicated that the social congruence of the tutor influences the learning process in a more significant way as compared to cognitive congruence and subject-matter expertise. This implies that the willingness of a tutor to establish an informal relationship with the students and display an attitude of genuine interest has the greatest impact on the progress made by students during the PBL process. Although a significant effect on the PBL process was not identified for cognitive congruence and subject-matter expertise, the impact for each of the independent variables on students’ achievement mirror the findings made by Schmidt and Moust (1995), which concluded that these tutor-related behaviours are all determinants of learning in a PBL curricula.

So, why is the impact of social congruence on the learning process so pervasive? During the process of constructing new knowledge and solving the problem, students would challenge and analyze possible solutions that are raised by peers while the tutor observes student interactions and encourage various kinds of cognitive activities, such as making connections between concepts and providing feedback (Dolmans et al. 2002). In addition, tutors should allow students to propose their own hypotheses regardless of whether they are inaccurate or superficial. It has been demonstrated that through the process of expressing their own thoughts, students would be able to identify their misconceptions and see how it fits with the correct knowledge (Schmidt et al. 2009). In order to create a learning environment where there is a free flow exchange of ideas, it is vital for students to feel comfortable in expressing their opinions openly. Therefore, the social congruence of the tutor can be anticipated to influence the learning process as a more socially congruent tutor would possess the interpersonal qualities to relate informally with students and this creates a non-threatening learning environment (Schmidt and Moust 1995). Furthermore, as learning in a PBL environment is believed to be cumulative whereby knowledge is built upon that which was gained in the previous learning phase (Yew et al. 2010), the amount of knowledge acquired during the learning process would in turn have an effect on students’ achievement, which was observed during data analysis.

Although a significant influence on the PBL process of cognitive congruence and subject-matter expertise was not found in this study, it is unlikely that these tutor-related behaviours do not affect the PBL process but only the outcomes of the process. In addition, the essay test that measured students’ achievement was administered immediately after the reporting phase, which did not give students extra time outside of the classroom for self-study. Thus, the knowledge gained must have been covered during the various learning phases of the PBL process within the same day, as also witnessed by the correlations between process and outcome which were highlighted in the results.

A possible reason that a statistically significant effect on the PBL process was not observed for cognitive congruence and subject-matter expertise could be due to the differential sensitivity of the measurement tools used in this study. The concept recall test required students to recall relevant keywords at the end of each learning phase and the number of keywords that could be easily recalled may have been limited. As students read and evaluate information from various resources, they may have understood the concepts but may not have paid close attention to the keywords used. On the other hand, the essay test required students to demonstrate their understanding of the topic and they were given the freedom to choose the words to describe what they have learnt. Hence, the essay test used to measure students’ achievement may have been more sensitive in picking up differences as compared to the concept recall test that was used to measure students’ learning process. Furthermore, the effects of cognitive congruence and subject-matter expertise on students’ learning process may have been too small to be detected by the concept recall test. A limitation with the concept recall test was that it only required students to list as many keywords as possible that were related to the discussion topic at the end of each PBL phase without having to make connections with the different concepts. This may have affected the analysis of students’ understanding about the concepts at each learning phase in terms of depth and accuracies (Yew et al. 2010). Nevertheless, the concept recall test was adequate in capturing a significant effect made by social congruence on the PBL process, which may have been a larger effect as compared to the impact made by cognitive congruence and subject-matter expertise.

Another possible explanation could be due to the use of natural variations as the study was conducted in a real school setting. For instance, the tutors who participated in this study were randomly chosen instead of being selected based on their specific profiles. In addition, the tutors had to be tutoring students who were taking the same subject, which limited the sample size and resulted in a limited number of eligible tutors. This led to a situation in this particular sample of facilitators that the standard deviation for social congruence was almost twice as large as those of cognitive congruence and subject-matter expertise (see Table 1). Limited variability in combination with somewhat reduced sensitivity of the dependent variable may explain the absence of effects of cognitive congruence and subject-matter expertise on the PBL-process in this study. Therefore, a larger sample size of tutors with greater variation in the levels of cognitive congruence, social congruence and subject-matter expertise is required before a more definite conclusion on the tutor-related behaviours on students’ learning process and achievement can be made.

The difference in PBL methodology practiced at this polytechnic as compared to other educational institutions limits the generalizability of the findings. Students at this polytechnic complete the PBL process from problem analysis to reporting phase within a day and students have close contact with their tutors throughout the day. However, the PBL process at other institutions may last for a longer period of time and the tutor may not be present at all times. These differences may influence the effect of tutor-related behaviours on the students’ learning process and achievement. Additionally, there is an absence of a long-term perspective in this study as both outcome measurement tools were administered on the same day immediately after the learning process. Further studies to include longer term assessment would have been beneficial to provide insights on the long-term effects of the tutor-related behaviours on student learning.

Conclusion

The effect of tutor-related behaviours on the PBL process and outcome was explored in this study. Our results indicated that social congruence had a significant influence on the learning process while social congruence, cognitive congruence and subject-matter expertise all had significant effects on student achievement. These findings are not only supportive of work previously done by Schmidt and Moust (1995) that advocate the positive influence of tutor-related behaviours on student achievement but provide new insights on their effects on the PBL learning process. Therefore, this study concludes that an attempt to improve the learning process and achievement of students in a PBL curriculum can be based on the development of effective tutor behaviour.

Besides possessing the necessary subject-matter expertise, tutors should recognize the importance of developing the ability to establish informal communication with the students as well as utilizing language that is easily understood by the students in the classroom. These qualities of the tutor contribute to creating a learning environment where students feel liberated to share their ideas and in developing strong tutor–student relationships that aid in promoting student engagement in discussions, which translates into better student performance during the learning process and at the end of the PBL process.