Background

Evidence-based medicine (EbM) is a prerequisite for decision-making in healthcare. All over the world, healthcare institutes follow the principles of EbM when reviewing and assessing the evidence for healthcare decision-making [1, 2]. The paradigm shift towards EbM challenges the methodological skills and attitude of healthcare professionals. A consumer-centred, evidence-based treatment requires basic EbM skills and scientific literacy [3, 4].

The transfer of evidence into routine care is often not optimal [5]. Barriers that impede the implementation of EbM have been extensively researched. Lack of time to put EbM into practice, false beliefs about EbM, insufficient support in the clinical setting and limited critical appraisal skills are the barriers that healthcare professionals most often face [6, 7].

There is empirical evidence that EbM training activities improve knowledge and skills needed for the critical appraisal of scientific papers [8]. Traditionally, training and continuing education in EbM focus on physicians. An early systematic review by Shaneyfeld and colleagues identified 104 trials on EbM training courses; n = 91 addressed physicians [9]. However, during the last few years curricula and textbooks aiming to achieve EbM competencies for allied healthcare professionals have been developed. Training courses in EbM skills for nurses, diabetes educators and other professionals associated with health have been shown to be feasible and well appreciated [10, 11]. Organizations like the Joanna Briggs Institute or the Centre of Evidence-Based Physiotherapy, which provide a variety of EbM workshops and learning opportunities for nurses, physical therapists, midwives, medical and allied health researchers, have been established. EbM resources have been made accessible, for instance, a free database of randomised trials, systematic reviews and clinical practice guidelines in physiotherapy. To address the increasing teaching demand and the need to improve the effectiveness of EbM, train-the-trainer courses have been developed [12]. Training in EbM for patients and consumer representatives is offered by some organisations [13, 14].

A variety of different approaches exists for teaching and learning EbM, for instance, by attending courses, conferences, workshops or journal clubs [15]. Recent systematic reviews showed inconsistent effects regarding the effectiveness of different EbM teaching and learning methods. Coomarasamy and Khan evaluated the effects of stand-alone versus clinically integrated teaching in EbM on several outcomes in postgraduates [16]. Stand-alone teaching was defined as classroom teaching, either didactic, interactive, or mixed. A total of 23 randomised controlled, controlled clinical as well as twelve before-after trials were included in this systematic review. Knowledge was assessed in 17 trials, critical appraisal skills in nine trials, changes in attitudes in six, and behavioural change in 14 trials. None of the trials evaluated clinical health outcomes. Stand-alone teaching as well as integrated teaching was effective in improving EbM knowledge but only clinically integrated teaching improved skills, attitudes, and behaviour [16]. Outcomes were predominantly determined by self-assessment. In contrast, a recently published Cochrane review on the effectiveness of training conducted to increase the “critical appraisal” skills only included trials if the assessment of outcome measures was based upon standardised and reliable instruments (e.g. tests, questionnaires). Three randomised controlled trials with 272 participants fulfilled the inclusion criteria. While a statistically significant improvement in participants’ critical appraisal knowledge was seen in two of the three trials, none of the three trials evaluated the process of care or patient outcomes [17].

Former systematic reviews dealing with EbM training activities have not taken the complexity of educational interventions into account. Complex interventions typically comprise interacting elements that are also influenced by contextual factors [18]. Educational interventions are often heterogeneous in their underlying theory, the methods used, the format and intensity, and the target population. The development of educational interventions requires great investment in testing procedures for their feasibility and acceptability prior to large-scale evaluations. The process of implementation should ideally be carefully prepared and piloted and it is recommended to take the whole chain of complex intervention development and evaluation into account while reviewing complex interventions. Thus, a non-customary approach for evidence synthesis is needed. The UK Medical Research Council (MRC) has provided the framework for understanding and appraising complex interventions [18–21]. All relevant patient outcome parameters need to be assessed, interdependencies between active elements should be taken into account, and all trials referring to development, evaluation and implementation of the educational intervention should be considered. The theoretical basis and its influence on the education program should be analysed. Adequate reporting of the elements and interdependencies within complex interventions are the prerequisite to interpret the outcomes.

To the best of our knowledge, a systematic review on EbM training for healthcare professionals with respect to the complexity of the educational interventions has not been performed. Therefore, the objective of this review is to evaluate the effects and the quality of the reporting regarding development and implementation of EbM training for healthcare professionals.

Methods

The review protocol has been registered in PROSPERO International Prospective Register of Systematic Reviews (crd.york.ac.uk/prospero/index.asp Identifier CRD42014013579).

Types of trials

We included all individual and cluster-randomised controlled trials (RCTs), controlled clinical trials (CCTs) as well as before-after trials. Non-blinded trials were included in the review, since blinding of participating healthcare professionals seemed to be unrealistic. The publication language was restricted to English and German.

Types of participants

Healthcare professionals in any clinical or academic setting were included. Trials on EbM training solely for physicians, medical students, patients and patient representatives, managers or purchasers were excluded.

Types of interventions

This review focuses on educational interventions aimed at improving EbM knowledge, skills, attitudes and behaviour in healthcare professionals. The interventions might cover the following contents: Formulating questions that could be answered by a systematic literature search; performing a systematic literature search; critically appraising selected publications; communicating trial results to consumer and patients.

We excluded trials that investigated the effects of teaching solely biostatistics or search strategies, programs focusing on specific health problems, medical education in general (not EbM in particular), and trials testing the effectiveness of implementing evidence-based guidelines.

Types of outcome measures

  1. 1.

    Outcome measures were attitudes, knowledge, skills, and behaviour regarding EbM which were objectively assessed through validated instruments.

  2. 2.

    Impact of EbM training on the implementation of EbM in routine care and patient-relevant outcomes like mortality, morbidity, and quality of life were assessed.

Search methods for identification of trials

The literature search strategy followed the Cochrane Handbook for Systematic Reviews of Interventions, version 5.1.0. [22]. MEDLINE, EMBASE, CINAHL, Cochrane Library, ERIC, Campbell Library and PsycINFO were searched systematically in September 2014.

The following terms were used: “Health Personnel (MeSH term)”, “dietician”, “dietitian”, “diabetes educator”, “evidence-based medicine”, “evidence-based nursing”, “evidence-based practice”, “evidence-based”, “journal club”, “critical read*”, “critical appraisal”, “science literacy”, “health literacy”, “risk literacy”, “education (MeSH term)”, “train*”

Reference lists of published reviews and included articles were checked for additional trials. If the full text was not available, the authors of the trials were contacted.

Selection of trials

Two review authors (LH, SB) independently assessed titles and abstracts from the search. Eligible articles were assessed for inclusion. Disagreement was solved by consensus.

Data extraction and management

Based upon CReDECI [23] and the CONSORT statement [24], we developed a standardized data extraction form that included information on the development, evaluation and implementation of complex interventions:

  • Description of the intervention

  • Description of the intervention’s development (e.g. theoretical and/or evidence base)

  • Information on pilot testing

  • Delivery of the intervention (who, how often, how long?)

  • Description of the implementation strategy

  • Description of any material and method used

  • Method of assessing participants’ preferences/interests/experiences

  • Description of process evaluation

  • Information on costs/resources needed for the implementation of the intervention

  • Description of what has been offered in the control group.

Data were extracted by two independent reviewers (LH, SB) and checked for accuracy. In case of discrepancy, the third review author (GM) was called in to reach consensus. Quality criteria following the Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [22] were applied in order to assess the risk of bias of included trials. Critical appraisal of trials was carried out by two independent reviewers. In the case of unclear or missing information, the corresponding author of the trial was contacted.

Since we found pronounced methodological heterogeneity, the trial results are presented in a narrative form only.

Results

A total of 14.507 articles were identified, of which 61 were considered for inclusion. After screening the full text articles, a total of 13 trials were included: four randomised controlled trials [25–28], two controlled clinical trials [29, 30] and seven before-after trials [11, 31–36]. Sample sizes ranged from n = 30 [26] to n = 168 [29] with a total of 1,120 participants. The reasons for exclusion are reported in the flow diagram (Fig. 1). In seven trials, participants’ age was not reported [11, 27, 29–31, 33, 34]. In the remaining trials, the age ranged from 18 to 64 years.

Fig. 1
figure 1

Flow diagram

Full size image

Trials were conducted in the United States of America [27, 30, 32, 35], the United Kingdom [26], Australia [31, 33, 34], Canada [36], Germany [11], Taiwan [29], Philippines [28] and Iran [25].

Healthcare professionals involved in these trials were predominantly nurses or nursing students, nurse managers, occupational therapists, physiotherapists, speech pathologists, dieticians and diabetes nurses or diabetes instructors with or without academic background and varying job experience.

The total trial duration ranged from eight weeks [35] to 27 months [32]. The characteristics of the included trials are summarized in Table 1.

Table 1 Characteristics of included studies (n = 12)
Full size table

EbM training courses

The EbM training offered varied in duration from courses lasting five hours [26] up to courses with 48 h [34] of teaching.

Bennet et al. [31] and Yost et al. [36] included all five core elements of EbM training in their program: 1) ask a question that can be answered; 2) identify appropriate sources for searching relevant information and perform a systematic literature search; 3) critically appraise selected publications on key elements; 4) implement EbM in everyday clinical practice; 5) communicate trial results to patients and consumers. Meyer et al. [11], Dizon et al. [28], Lizarondo et al. [33], McCluskey et al. [34], Varnell et al. [35], Courey et al. [30] and Levin et al. [27] included the first four elements, while Stevenson et al. [26], Chen et al. [29] and Jalali-Nia et al. [25] included only the first three elements. The report by Kim et al. [32] did not provide a full description of the training’s elements.

Eight training programs [11, 25, 26, 29, 31, 33, 35, 36] offered classroom-based activities for teaching the principles of EbM. Five trials [27, 28, 30, 32, 34] included co-intervention in addition to classroom teaching, such as mentorship in participants’ homes or institutions, online support, email lists to facilitate communication or presentation of relevant literature in clinical settings.

A description of EbM training programs is provided in Table 2.

Table 2 Description of EbM training programs
Full size table

Risk of bias in included before-after trials

Details of risk of bias in the included before-after trials are displayed in Table 3.

Table 3 Risk of bias summary for the included before-after trials
Full size table

The trials by Meyer et al. [11], Varnell et al. [35] and Lizarondo et al. [33] had the lowest risk of bias, while Yost et al. [36], Kim et al. [32] and Bennet et al. [31] had the highest risk of bias.

Risk of bias in included controlled and randomised controlled trials

Details of risk of bias in the included controlled and randomised controlled trials are displayed in Table 4. From the included controlled trials, the trial by Courey et al. [30] had a higher risk of bias than the trial by Chen et al. [29]. From the included randomised controlled trials, the trials by Dizon et al. [28] and Levin et al. [27] had the lowest risk of bias, while the trial by Stevenson et al. [26] had the highest risk of bias.

Table 4 Risk of bias summary for the included controlled and randomized controlled trials
Full size table

Intervention effects

Details of intervention effects and applied assessment instruments are displayed in Additional file 1.

Impact on attitudes

Two before-after trials [31, 32] reported that attitudes towards evidence-based practice did not significantly improve after EbM training, while one trial [35] reported higher scores on the attitude scale at the end of the program. Lizarondo et al. [33] examined the impact of EbM training on the attitudes of different associated healthcare disciplines. A significant improvement in attitudes was seen only for physiotherapists, but not for speech pathologists, occupational therapists, social workers or dieticians/nutritionists.

The controlled clinical trial by Courey et al. [30] reported a decrease in positive attitude from pre- to post-test in the intervention group and no change in the control group.

In a randomised controlled trial, Jalali-Nia et al. [25] showed a statistically significant difference between the intervention and the control group with higher scores for positive attitudes in the intervention group. Dizon et al. [28] demonstrated significantly increased positive attitudes in the intervention group immediately post-training and three month post-training. The results obtained by Stevenson et al. [26] also demonstrated small effects on positive attitudes towards the EbM concept. However, the reporting in the original paper was somehow inconclusive and a request to the authors remained unsuccessful. The specific affected element remains unclear. In the randomised controlled trial by Levin et al. [27] a statistically significant improvement in the intervention group compared with the control group was seen twice: first after the 16-week educational and mentored intervention period and secondly (nine months later) after an evidence-based practice (EbP) implementation project.

Impact on knowledge and skills

Five before-after trials tested whether EbM training leads to an increase in EbM-related knowledge. Three trials additionally tested whether knowledge in EbM is influenced by training. Bennet et al. [31], Meyer et al. [11], and Lizarondo et al. [33] documented that participation in the course was associated with a significant increase in knowledge directly after the training. Meyer et al. [11] also observed increased skills in EbM. McCluskey et al. [34] collected data at baseline, post-training and eight months later. There was a significant increase in knowledge when baseline and post-workshop scores were compared. Improved knowledge scores were maintained after eight months. Yost et al. [36] collected data at baseline, post-training and six months later. Knowledge and skills increased significantly from baseline to post-training measurement and from baseline to six-month follow-up. The post-training measurement compared to that at six months follow-up showed a significant decrease in knowledge and skills. In their controlled clinical trial, Chen et al. [29] showed significantly increased scores for knowledge and skills in the intervention and in the control group. However, the mean score in the intervention group was significantly higher than the mean score of the control group. The randomised controlled trial by Dizon et al. [28] found significantly increased scores for knowledge and skills in the intervention group compared with a waitlist control group directly post-training and at three months follow up.

Impact on EbM implementation and patient-relevant outcomes

Three before-after trials and two randomised controlled trials tested whether EbM training leads to an increased EbM implementation. Lizarondo et al. [33] tested the impact of EbM training on different associated healthcare disciplines. Physical therapists, social workers and dieticians/nutritionists showed statistically significant, positive changes in EbM implementation scores but speech pathologists and occupational therapists did not. While Varnell et al. [35] also reported higher scores on EbM implementation scales at the end of the program, Yost et al. [36] found no significant increase in EbM implementation behaviours from baseline to six-month follow-up. Levin et al. [27] demonstrated in a randomised controlled design more EbM implementation behaviours in the intervention group than in the control group at the end of the training. In the randomised controlled trial by Dizon et al. [28] improved EbM implementation behaviours were seen in the intervention group but not in the control group.

None of the trials assessed patient-relevant outcomes.

Reporting quality with regard to the development and implementation of a complex intervention

A total of eleven trials described the elements of the applied training in detail. However, reporting quality varied regarding the description of the interventions’ development. Most of the trials provided insufficient information concerning piloting, description of the implementation strategy and the methods of assessing participants’ preferences, interests and experiences. Almost no information was provided on process evaluation and costs or resources needed for implementation. Details of the reporting quality are shown in Table 5. In summary, the trials by Dizon et al. [28], Levin et al. [27], Varnell et al. [35] and Meyer et al. [11] showed the highest quality of reporting, while Jalali-Nia et al. [25], Chen et al. [29] and Courey et al. [30] demonstrated poor reporting quality on the development and piloting of the complex intervention.

Table 5 Reporting quality with regard to the development and evaluation of a complex intervention
Full size table

Discussion

The results of this systematic review on the effectiveness of EbM training show some impact on the knowledge and skills of healthcare professionals. However, the improvement in knowledge and skills was often rather small. Three trials demonstrated significant higher scores on EbM implementation scales and one trial reported improved EbM implementation behaviours measured by activity diaries. However, improvement of EbM implementation was self-perceived; hence, the impact on the practical application of EbM remains unknown.

Adequate knowledge and skills are indispensable for successful implementation of EbM but are not the only prerequisite. Negative attitudes, low management priority and no willingness to change current practice models are well-described barriers of EbM implementation [37].

Conflicting results were seen regarding the impact of EbM training on attitudes towards EbM. While some trials reported improvement, other trials did not. One trial even reported a decrease in positive attitude from pre- to post-intervention in the EbM training group. The trial by Lizarondo et al. [33] demonstrates inconsistent outcomes across different disciplines of healthcare professionals following EbM training. Some disciplines showed statistically significant improvements in all outcomes; others did not. It is unclear whether the training programs evaluated in this review will be comparably effective across all branches of healthcare professionals. Current systematic reviews also conclude that teaching interventions may positively influence EbM-related knowledge, skills and attitudes in healthcare professionals [17, 38].

These reviews did not make any attempt to take the nature of complex interventions into account, neither did they refer even to suggested models like the UK MRC framework.

EbM training for healthcare professionals as complex interventions comprise different elements that act interdependently, e.g. train-the-trainer modules, number of sessions, curriculum, corresponding media and materials. There are also different contextual factors, such as setting and didactic strategies, as well as the educational and professional background of the participants that may influence the intervention effects.

The majority of the trials offered exclusively classroom-based EbM training, while some trials included co-intervention in addition to classroom teaching. There is much debate about what is the best type of educational activity to achieve a substantial increase in putting EbM into practice [39]. Empirical evidence exists in favour of clinically integrated EbM training over classroom-based teaching in relation to changing behaviour [8, 16, 40] which might be an indicator for successful implementation.

The process of developing a complex intervention has several phases. Not all research will need to begin at the beginning and work stepwise through an entire framework for example provided by the MRC. Sometimes evidence already exists; sometimes steps are more or less important [18]. However, complete reporting on the development and piloting of all components is most important in order to interpret the outcomes of a complex intervention. The trials included in our review provided insufficient reports on the development and implementation of the EbM training. Piloting was rarely reported by the randomised controlled trials. However, avoiding proper piloting might lead to non-effective interventions since no attempt was made to understand and reduce procedural, clinical, and methodological uncertainties in advance of the implementation of the intervention within the main trial. There was also insufficient information provided about participants’ preferences, interests and experiences. This might result in low acceptability of the intervention. Unfortunately, almost no information was provided about process evaluation and the costs or resources required. Thus, no insight was given into why an EbM training might have unexpected outcomes, why a successful training worked or how it might be optimized. Information about costs and resources is important for decision-makers.

Our review has several strengths. In order to increase the validity of the results only trials using validated assessment instruments have been included. Our review is the first taking the complexity of the included interventions into account. Therefore, we applied a specific criteria list on the quality of reporting of complex interventions [23]. Reporting a complex intervention trial according to the requirements of CReDECI might improve transparency and understanding of the intervention and might also have an impact on the value of future systematic reviews dealing with complex interventions

A full description of the EbM training and an understanding of its elements are crucial for the reproduction of the intervention’s evaluation, the adaption of EbM training to different settings, and for long-term implementation [41, 42].

Our review has some limitations. Of the seven contacted authors, only two replied to our request. Since we considered only English or German language publications for inclusion, a language bias could not be ruled out.

Conclusions

There is insufficient evidence that available EbM trainings for healthcare professionals are likely to result in a meaningful change in EbM behaviour. Future trials should not focus only on participants’ knowledge, attitudes and skills. EbM training is also supposed to foster healthcare professionals’ use of EbM. Relevant outcomes should be used to assess the effectiveness of EbM training and to investigate whether such courses lead to changes in care processes or patient-relevant outcomes.

In this review, conclusions about effective training elements cannot be made due to the poor reporting quality of the included trials. In order to generate formats suitable for long-term implementation, future EbM training should be carefully developed, theory-based, piloted and finally investigated in a robust, randomised controlled trial.