Background

Clinical trainers in general practice (GP) participate in continuing professional development (CPD) and faculty development (FD) courses at training institutes for GP speciality training. These educational interventions are expected to improve the quality of trainee-education in clinical practice by way of a cascade effect, representing the transfer of information from course to clinical trainer (Step 1) to training practice (Step 2) and to trainee (Step 3).

This is important because, during their clinical training [1], trainees are expected to grow into their future role as independent and competent professional physicians capable of providing high-quality patient care [2]. They learn clinical reasoning skills and behaviour not only at special teaching moments where the clinical trainer functions as a teacher and mentor, but also during everyday practice when they observe their clinical trainers as role models in the clinical workplace [3–6]. Stegeman emphasizes that role modelling is a major means of clinical workplace learning. [7] Bandura, in his social learning theory, explains learning by observation through modelling (Fig. 1) [8]. By adding the step of ‘apperception’ (the process whereby the perceived qualities of the trainer are related to past experiences) to this process, trainees can assess the role model behaviour of their trainers more consciously (Fig. 1) [9].

Fig. 1
figure 1

Model based on the social learning theory according to Bandura, supplemented by the steps of apperception (by using the RoMAT) for the trainee and amplification of the role model behaviour for the trainer

Full size image

In order to ‘amplify’ their role model behaviour, clinical trainers need to develop role modelling awareness [10] and to be explicit about what they are modeling. [7] To prepare clinical trainers for their roles as teachers and role models, they receive structural training in both medical content and didactic topics. It is unknown, however, whether both these elements are conveyed by the trainers to their trainees, and which characteristics of the trainer influence this process [11]. It is known that ‘teach-the-teacher’ courses focusing on didactic skills result in better teaching/coaching behaviour, and are therefore beneficial for trainees [12–14]. Consequently we may expect that training for clinical trainers [15–19] that focuses on knowledge and attitudes regarding a medical subject, with training in conveying knowledge and an appropriate professional attitude to the trainee as a positive role model [6], may also improve the professionalism of the trainer and how this is conveyed to the trainee.

The research questions of this study were therefore the following:

  1. (a)

    Does a teach-the-trainer course addressing the knowledge, attitude and the role model function of the trainer lead to an improvement in the knowledge and attitude of both the trainer (Step 1 of the cascade) and the trainee (Step 3), and in the role model behaviour of the trainer (Step 2)?

  2. (b)

    Which characteristics of the trainer/trainee are related to larger improvements in knowledge, attitude, or role modelling scores? (Step 1 and 2)

  3. (c)

    Does positive role model behaviour by the trainer affect the performance of the trainees? (Step 3)

Methods

This study was performed as a controlled intervention study, with a training course serving as the intervention.

Context

GP trainers in the Netherlands spend 4 days a week supervising GP trainees in their own practices during the first and third years of the three-year GP training programme. In their second year, trainees work in a variety of clinical settings, supervised by clinical trainers. Trainees spend one day a week, and trainers 8 days a year at one of the eight institutes for GP speciality training, where they are taught by GPs and behavioural scientists (both to be referred hereafter as ‘teachers’). This ‘training for trainers’ consists of half-day or full-day courses that aim to improve the competences of the trainer, thereby assuming to improve the clinical training given to trainees.

Participants

GP trainers and their trainees at four institutes for GP speciality training participated in this study. The intervention groups were located in Amsterdam (mandatory participation) and Nijmegen (voluntary participation), and the control groups in Leiden and Maastricht. The participants were approached between January and September 2012.

We included the control groups to correct for any changes in knowledge and attitudes attributable to publicly available information published during the study period.

All participants were informed about the nature of the study, that participation in the study (not training) was voluntary, and that questionnaires would be coded in order to prevent responses from being traceable to individual respondents. The trainers and trainees signed informed consent forms. These forms and other identifiers were stored in a locked cabinet by the head of the research department (MWdW). Ethical approval for this study was obtained from the Ethical Review Board of the NVMO (Dutch Association for Medical Education).

Intervention

We organized a combined CPD/FD course introducing a new self-management method for the treatment of obesity.

The aim of the training course was, firstly, to acquire knowledge about obesity and its treatment, secondly, for participants to become aware of their own attitude toward overweight patients, and, thirdly, to emphasize the importance of conveying knowledge and the correct attitude to trainees by serving as positive role models. The training took place in groups of 15–20 trainers and consisted of 3 h of education at the start of the intervention and 3 h, 3 months later. The duration was the same as standard GP training courses in the Netherlands, taking into account the limited available time for CPD due to the demanding responsibilities of general practice in terms of patient care. However, several previous studies had shown that 6-h teaching skills courses significantly improved residents’ teacher ratings [20, 21].

The training course comprised interactive presentations, peer discussion about case histories and statements, self-reflection regarding attitudes and exercises aimed at practising how to convey professional competencies to trainees. All participants received the necessary materials, such as booklets for patients, to enhance implementation in daily practice. We used these mixed didactic and interactive training sessions because a previous review showed that these can improve professional practice and health care outcomes, provided the focus is on outcomes that are likely to be perceived as serious [22].

We chose the subject of weight management because of its importance in preventing health problems in patients [23, 24]. However, GPs are still unsure about implementing this policy. Trainees therefore experience serious difficulties in becoming competent in this subject since they are influenced by the negative role model behaviour of clinical trainers [25]. New guidelines for GPs in the Netherlands were published on this subject shortly before the intervention [26], and a new evidence-based [27] intervention was introduced for weight management in GP practice (the minimal intervention strategy for obesity [28]), addressing the barriers to implementing weight management [29] using booklets for patients. All this led to the assumption that an intervention focusing on weight management would fulfil the learning needs of trainers and trainees, and could lead to significant improvements in terms of knowledge and attitudes, and possibly bring about changes in behaviour in this area.

Outcome measures

To evaluate the study we used questionnaires on:

  • Characteristics We asked the participants to record the characteristics of their practices and themselves, especially their weight and weight-change in the previous year, as these influence motivation and self-efficacy in treating patients with obesity [30–32].

  • Knowledge We designed a questionnaire consisting of 40 multiple-choice questions (the minimum for a reliable MCQ exam [33]) on weight management. We designed three equivalent questionnaires to ensure that each participant would complete an alternative list of questions every time. The questions were based on GPs’ guidelines on obesity [26, 34–36], and reviewed by two experts.

  • Attitudes We translated an instrument measuring attitudes toward weight management, using forward–backward translation. This self-assessment instrument, consisting of 20 items scored on a 5-point Likert scale, with higher scores representing a more positive attitude, has not been formally validated, but has already been used successfully in previous studies [37, 38] (see Table 5 in Appendix 1).

  • Role modelling We developed and validated a tool for assessing the role model behaviour of clinical trainers: the RoMAT [39] (see Table 6 in Appendix 2). This tool was developed on the basis of a systematic review of the literature [9] aimed at identifying the attributes of good role models. It consists of 17 items scored on a 5-point Likert scale, split into two components: ‘Caring Attitude’ and ‘Effectiveness.’ Both components include an equal number of items addressing personal, teaching, and clinical qualities, with high reliabilities (Cronbach’s alpha 0.92 and 0.84, respectively).

    To evaluate the extent to which the trainers were aware of their role model behaviour, we also asked the GP trainers to score themselves, and the trainees to score their trainers as role models using a 5-point Likert scale.

  • Behaviour We asked trainers and trainees whether they discussed the subject of the intervention and whether they used the booklets in their practice as an objective measurement of implementation in daily practice.

Assessment (Fig. 2)

The GP trainers at the intervention institutes completed questionnaires on knowledge and attitude three times: before the intervention (T1), immediately after the intervention (T2), and three months after the intervention (T3). The GP trainers at the control institutes completed these questionnaires only at T1 and T3, since no change was expected in the scores at T2 without an intervention. All the trainees completed two questionnaires assessing their own knowledge and attitude, as well as the role model behaviour of their clinical trainer, at T1 and T3. Trainers and trainees answered the questions on practice behaviour only at T3.

Fig. 2
figure 2

Schematic representation of the assessment

Full size image

Analyses

Data analysis was performed using SPSS version 20 (IBM SPSS Statistics, IBM Corp., Armonk, NY, USA). We used descriptive statistics to report the characteristics of the participants and training practices.

  1. (a)

    We analyzed the change in scores in the intervention groups with a repeated-measures design, and compared the scores of the intervention and control groups using a one-way ANOVA. We calculated the effect sizes for all comparisons (ES = Mi − Mc/SDc) [40].

  2. (b)

    We calculated the correlation coefficient for the question about being a positive role model.

  3. (c)

    Using one-way ANOVA, we analyzed the influence of the characteristics of the trainer on the increase in their scores and we did the same for the scores of the trainees.

  4. (d)

    We investigated the relationship between the role model behaviour of the trainers and the individual test performances of all trainees in the intervention group (both the mandatory and voluntary ones) using Pearson’s correlation coefficient r at T3. Using univariate linear regression, we analyzed the influence of role model of the GP trainer on the T3 scores of the trainees.

Finally, to investigate the two items on behaviour, we used McNemar’s test for the question of discussing the subject and descriptive statistics for the question of distributing the booklets.

Results

Response

Only the GP trainers who attended the central curriculum days participated, and not every participant was present on all days: 184 (75 %) GP trainers participated in the mandatory group, 25 in the voluntary group, and 171 (67.1 %) trainers in the control group. The characteristics of the respondents and their practices varied in each group (Tables 1, 2).

Table 1 Personal characteristics of the respondents
Full size table
Table 2 Characteristics of GP training practices
Full size table

Improvements in knowledge, attitudes, and role model skills (Table 3)

In both intervention groups, knowledge scores increased significantly, and persisted to T3 (Step 1). Because of the small number of participants in the voluntary intervention group, their results were omitted from the comparative analyses. The knowledge scores of the trainees in the control group at T3 were significantly (p =  .02) higher than those of the intervention group, but the effect size (p = .37) was of little significance. [41] (Step 3)

Table 3 Mean scores on the questionnaires of trainers and trainees; score differences between T1 and T3 on knowledge, attitude and role modelling for each group; effect sizes for the differences between intervention and control groups
Full size table

There was no correlation (r = −.010, p = .94) between the rating of the GP trainer and the rating of the trainee on the general question of whether the trainer was a positive role model. Conversely, the ratings of the trainees on the role model question correlated positively with the scores of the same trainees on the ‘Caring Attitude’ (r = .54, p < .001) and ‘Effectiveness’ (r = .63, p < .001) components (Step 2).

Influence of the respondents’ characteristics

Although no clear pattern emerged from the data, there were a few significant effects of characteristics on the change in scores (ΔT3–T1) of the GP trainers (Table 4). GP trainers who had recently lost weight, improved their knowledge more than those whose weight was stable or increasing [30, 31], showing that a topic with personal relevance motivates learning. Also, less experienced GP trainers improved their attitude more than experienced trainers, while more experienced trainers improved their role model behaviour in the ‘Effectiveness’ component. By contrast, GP trainers who worked alone in their practices showed better role model behaviour in the ‘Caring Attitude’ component than GP trainers who worked in a practice with others. GP trainers with fewer patients in their practices improved more on both the ‘Caring Attitude’ and the ‘Effectiveness’ components, compared with colleagues with over 2,500 patients. No significant influence was found of the trainees’ characteristics on their scores.

Table 4 Influence of the characteristics—personal, as a trainer and as a physician—of the GP trainers on the changes in their scores before and after the training, compared with Bonferroni’s procedure
Full size table

Influence of role model behaviour on the trainees

There was a very small, but significant, correlation at T3 between the ‘Effectiveness’ score of the trainers and the attitude score of the trainees, r = .201, p = .027, indicating that GP trainers with better role model behaviour on the ‘Effectiveness’ scale were associated with GP trainees with the highest scores on attitude.

Regression analysis with role model behaviour as the independent variable and the attitude of the trainee as the dependent variable was significant, but explained only 3.2 % of the variance in the attitude of the trainee, B(SE) = 2.82(.23), p = .027, showing that high levels of ‘Effectiveness’ in the role model behaviour of GP trainers have a small effect on the attitude of their trainees (Step 3).

There was a discrepancy between trainers and trainees concerning the question whether the trainers discussed the subject of the training course (43.6 vs 15.9 %, p < .01). The trainers and trainees rarely handed out the booklets in daily practice (10.9 and 3.2 %) (Step 2).

Discussion

A combined CPD/FD course for GP trainers on weight management and their role model function, with the aim of transferring professional competencies as a physician, teacher and role model, resulted in knowledge gains among the GP trainers, even measured using an objective examination and pre-, post-, and delayed post-tests, when compared with a control group. (Step 1) The characteristics that influenced the role model behaviour of the trainers were whether or not they were experienced, the number of patients in the practice and whether the GP trainers practised alone or in partnership. (Step 2) Only a few GP trainers and a very low percentage of the trainees reported using the booklets, indicating restricted use of the information from the training course in daily clinical practice (Step 2), and there was no improvement in trainees’ knowledge or attitude. Finally, a small correlation was found between trainers with better role model behaviour on the items on the ‘Effectiveness’ scale and trainees with the highest scores on attitude. (Step 3)

Strengths and limitations of the study

The strength of this study lies in the evaluation of the intervention, which was based on an objective measurement of knowledge, completed in the presence of a teacher to ensure objectivity, and we evaluated the role model behaviour of the trainers using the scores of their trainees [12–14] Furthermore we measured the effects of the intervention on the trainees and we evaluated the implementation in daily practice by counting the number of booklets handed out.

There are also limitations to this study. Firstly, there is some doubt about the generalizability of the study. Generalizability was improved by conducting the study in four unrelated training institutes located in different parts of the country to minimize local influences. Although the number of participants in the voluntary intervention group was small, the results from this group showed the same pattern as the results in the mandatory intervention group, thus affirming the value of the properties found.

To reduce the chance of educational interventions at one institute affecting the results of our comparison, we used the GP trainers and trainees at two institutes as controls. With a response of 75 % in the mandatory group and 67 % in the control group, our group of participants seems fairly representative.

Secondly, we found differences in characteristics between the groups. These may have been due to the location of the practices.

Thirdly, a large number of GP trainers failed to follow-up because the attendance of trainers at central curriculum days fluctuated widely; they were often required to work in their practices, shared their presence at training days with co-clinical trainers, or had other obligations.

Finally, only 25 GP trainers chose weight management from a list of different topics and volunteered for the intervention. Previous studies have indicated that GP trainers are not convinced that the treatment of overweight patients is meaningful, which could explain this finding. [25, 43]

Comparison with existing literature

Several previous CPD/FD studies [15, 17–19] have integrated clinical content with learning about educational methods in voluntary courses for clinical trainers, using different teaching methods; all had positive results, including behavioural changes. Because of the clinicians’ reported lack of time, two courses were, like our training course, short (2–4 h) [15, 18]. The results of these studies were based on self-reporting after the course and three months later. (Step 1) None of these studies used more objective outcome measures or evaluated the course in the target population, (the trainees) or used control groups. As with our findings, one study showed that trainers did not use the written information distributed on the course for their own patients [15]. Another study showed that clinicians routinely overrated their skills and knowledge in self-evaluations, using retrospective pre-programme scores [19]. In our study there was no correlation between the judgment of the trainers regarding whether they were a positive role model and the scores given by their trainees on the RoMAT. There would appear to be a discrepancy between the trainers’ own assessment of their capacities and the observations of the trainees.

Consequently, our limited positive findings may be due to the fact that, instead of relying on the possibly overoptimistic self-evaluation by the trainers [44], we used a more objective means of assessment: objective measurements amongst trainers and trainees, measurements in the training practice, and evaluation of the trainers by the trainees. (Step 2)

Some of the characteristics of the trainers appear to influence role model behaviour. Being more experienced had an influence on the ‘Effectiveness’ scores, implying that being more experienced relates with the ability to meet the learning needs of trainees [3]. Meanwhile, trainers who work alone and were not distracted by colleagues in their practice showed better role model behaviour on the ‘Caring Attitude’ component, thus appearing to be more open toward their trainees [41]. Higher patient numbers in practices (i.e. busier physicians) reduced both scores, leading to less positive role model behaviour. Previous studies found that reserving time for the trainee is a requirement for positive role modeling [3, 42] (Step 2).

The aim of the CPD/FD course, of improving professional competences of the trainee through a cascade effect, was not achieved. Although the first step, that of improving the clinical, teaching and role modelling competences of the trainer, was partly achieved, only a small effect was found for the influence of positive role modelling, and no significant effect on practice behaviour, handing out the booklets. Various reviews [12–14] have discussed quality and composition as factors in the effectiveness of the CPD/FD course. Our study showed that there is a greater loss of competences after the first step, from trainer to training practice and to trainee.

The absence of any knowledge gains on the part of trainees could be explained by the discrepancy between trainers and trainees regarding the question whether the trainers discussed the subject of the training course, implying an overestimation by the trainer and therefore less of a focus on conveying knowledge or, even more importantly, poor receptiveness by the trainee if the topic was not of interest to them at that point in their training. However, the lack of improvement in attitudes toward weight management in our study was in line with previous studies that used the same questionnaire [37, 38] or the same line of questioning [45, 46]. This trend seems to imply that education and the acquisition of more knowledge do not necessarily lead to a better attitude [29]. Finally, although positive role modelling seems to be related to more positive attitudes among trainees, no improvements in role model behaviour could be established and thus no effect on the achievement of the trainee. This may originate from the discrepancy between the trainers’ and the trainees’ evaluation of the trainer as a positive role model. An overestimation of their own positive role modelling may have led to trainers not being motivated to improve their role model behaviour, but also shows the necessity of feedback from trainees on the role modelling of trainers (Step 3).

Conclusions

Our study analyzed the effects of a CPD/FD course for clinical trainers, by surveying the target population, the trainees. The course integrated knowledge, attitudes, and role modelling, and proved to be a first step toward improving the knowledge of the trainer, but did not result in improved overall professional outcomes for the trainees [47]. Some of the characteristics of the clinical trainer did influence their role model behaviour and there was a small correlation between more positive role model behaviour and positive attitudes among trainees.

Implications for future courses and research

More research is necessary to establish how to improve the effectiveness of train-the-trainer courses, encouraging clinical trainers to maximize their awareness of conveying knowledge and modelling the correct behavior [3, 4], as well as to determine how to maximize the professional gains for trainees [6] and evaluate such training objectively in the training practice and target population [13, 14].

Essentials

  • Integrating knowledge, attitude, and role model behaviour in one course leads to improvements in the knowledge of clinical trainers.

  • More research is necessary to establish how to improve the effectiveness of train-the-trainer courses in terms of professional gains for the trainee.

  • Effectiveness of courses needs to be assessed by means of objective evaluation among the target population and in daily practice.