Background

Using the best evidence for determining a patient’s therapy is important [1]. Traditionally, the ability to make treatment decisions was limited to a physician’s own experience with the problem, which carries the potential risk of error [2,3,4,5]. The value of evidence-based medicine (EBM), which involves using updated, relevant, and trustworthy evidence to inform medical decisions, has been broadly acknowledged [6]. As a result, teaching EBM has become crucial for medical students’ development into lifelong independent learners and critical thinkers that can offer high-quality patient care [7]. The research problem is that physicians—even senior ones—may miss some of the important nuances of EBM, which subsequently impacts the quality of their teaching [7]. Given the lack of EBM teaching aids, we considered whether an interactive e-learning tool using SimMAGIC software [8] could be used as an effective teaching aid for EBM. In this study, our aim was to use an interactive e-book or PowerPoint to evaluate instructors’ teaching effects on EBM.

Chiu reported that, according to a national survey in Taiwan, the knowledge, skills, and attitudes regarding evidence-based practice (EBP) improved among physicians and nurses after undergoing training [9], thus indicating that the practice of EBM is important and can be improved through training [10]. Using Internet information sources to answer patient-related questions has taken an ever more important place in the daily practice of physicians [11]. A practical new instrument for measuring good-quality clinical application is vital for educating teachers and evaluating the effect of their teaching [12].

Electronic learning (e-learning) differs from traditional educational methods in that it uses digital platforms for learning [13, 14]. Using technology for learning has the benefit of being more student-focused and flexible [15]. The learner becomes a more active participant in the acquisition of knowledge, as opposed to being a passive recipient [13]. Cook reported that e-learning performs on par with traditional classroom teaching [16]. However, other researchers have reported that e-learning is no better than traditional classroom education for improving the proficiency of novices [17]. In modern curricula, e-book technology is considered a useful supplement to traditional methods [18, 19]. However, no studies have yet indicated whether the teaching effect of a lecturer is better after learning from an online e-book than traditional learning.

Our aim was to use an interactive e-book or PowerPoint to evaluate instructors’ teaching effects on EBM in order to evaluate the effectiveness of a clinical teaching EBM e-book to improve teacher-related skills, as well as to foster both teachers’ and learners’ interest in studying EBM. One month later, we arranged for every teacher to teach an EBM class and to have students assess the effectiveness of their teaching. This method may be a promising and effective way to improve physicians’ EBM-teaching skills in the future.

Methods

Ethical process

This study was approved by the institutional review board of the ethics and clinical research committee of CGMH (104-9177B). All teachers and students provided their consent prior to participating in the study. To blind the statistician, the data was anonymized, and participants’ names were removed from the questionnaires before data analysis was performed. Data storage, participant recruitment, and data collection were all performed by a research assistant who had no assessment relationship with the participants.

Study population

This study was conducted from September 1, 2016 to October 31, 2017 (participants are shown in Table 1). Visiting staff teachers at Chang Gung Memorial Hospital (CGMH) were enrolled in this study, and two hundred teachers in the hospital were randomly selected by our group, choosing one for every 2.5 with the 500 names listed on our hospital’s computer. Those who did not consent to the assessment were excluded. The flow diagram of this study and its procedure is shown in Fig. 1. All teachers and students provided their consent before participation after understanding the study. A research assistant obtained the consent, and participants had the right to withdraw their data and consent at any time.

Table 1 Basic demographic characteristics of the teachers
Full size table
Fig. 1
figure 1

The procedures of the study and the flow diagram of the study selection of participants

Full size image

Tools: e-book or PowerPoint for teachers’ learning and teaching

Our team designed an online e-book using SimMAGIC software. The e-book contains an introduction to the basic concepts of EBM, EBM databases, database literature search skills, critical appraisal, and effectiveness evaluation methods using a repeatable answer-correct quiz [20, 21] (see availability of data and materials). The structure included EBM definition, EBM five steps, searching skills, RCT, systematic review, and four tests. The content for the e-book and PowerPoint was based on the EBM skills found on our hospital’s website at https://www1.cgmh.org.tw/library_s/EBMRdatabase.htm. (This cannot be accessed by Internet Explorer, but only Google Chrome, all rights reserved.)

A repeatable answer-correct quiz was provided to evaluate each of the learning objectives at the end of each topic. The learners can try the test multiple times to have better memory. Problem sets were created for readers and used for self-assessment purposes. They were presented in a multiple-choice question format, and readers could respond by clicking to answer the question. Another traditional method (PowerPoint) was designed for use as a control group (Additional file 1). The PowerPoint learning material was designed with the same contents as that of the e-book and also included the basic concepts of EBM, EBM databases, database literature search skills, critical appraisal methods, clinical application, and effectiveness evaluation. In the PowerPoint presentation, the questions were presented in a multiple-choice format but responses could not be provided by clicking.

Study setting

In order to provide teachers with a better method of learning EBM and then assess their teaching effects through students’ feedback, a flipped classroom model was designed with an e-book or PowerPoint for teachers’ learning. In order to decrease the influence of students’ characteristics, the effect of teachers’ instruction was assessed by the same three participating 7th year students. It is very difficult to assess teachers’ day of the week and level of fatigue. Nevertheless, the study was appointed before teachers entered the study so they could answer the Fresno test and have a teaching class, thus minimizing the effect of other potentially confounding factors.

The teachers created an answerable question using the PICO model from a true case in the hospital and searched the electronic databases available at CGMH prior to the study. They attempted to consolidate the evidence in order to reach a conclusion and make a recommendation for the question. We then gave them access to the e-book or PowerPoint presentation and encouraged them to practice using it. They re-examined the information they had written by searching, consolidating evidence, and correcting the recommendation. Afterward, the assessment questionnaires shown in Table 2 were completed. The study group utilized the e-book, while the control group used the PowerPoint presentation without the e-book.

Table 2 Modified Fresno test in the e-book and PowerPoint groups (mean±SD)
Full size table

A randomized controlled trial was established with a 1:1 allocation ratio to assess the teachers before they were assigned to either the e-book group (n = 40) or the control group (n = 40). Randomization was carried out (allocation concealment) through central randomization performed by an independent randomizer (assistant). Random assignment with a two-blocked design in the order of their entry into the study was adopted using http://www.randomizer.org. The teachers were blinded to the purpose of the study.

We employed an evaluation method using a structured paper questionnaire of the Modified Fresno test (Table 2) to investigate our teachers’ EBM skills both before and after the e-book or PowerPoint learning. The Fresno test questionnaire included three dimensions (knowledge, search abilities, and critical appraisal skills), with a total of 12 items (Table 2). The scale for each item was 0–10, the total scale for one Fresno test was 120. Teachers then practiced the EBM steps to search, critically appraise, and analyze the level of evidence for the article they found, make a recommendation, and apply it to a clinical situation.

One month later, each teacher gave a lesson on EBM according to the e-book (see availability of data and materials) or PowerPoint content shown in the S1 Appendix.

Student assessment of the teachers’ EBM teaching effect

One month later, each teacher gave a 20 to 30-minute lesson on EBM regarding the formulation of problems, database search techniques, and literature appraisal for three students, who then assessed their teaching skills and ability. We provided eight statements for the three students for each teacher that they could answer with the following options: strongly agree, agree, neither agree nor disagree, disagree, and strongly disagree. We converted the scale into Likert scale with strongly agree 5, agree 4, neither agree nor disagree 3, disagree 2 and strongly disagree 1. The eight statements were as follows: “I am satisfied with this lesson,” “The teaching was of high quality,” “The teacher had a good attitude,” “This was a good teaching method,” “It helped my understanding,” “I was allowed to ask unlimited questions,” “The teacher listened to my questions,” and “It aroused my interest in EBM.”

Validity

The content validity of the EBM skill questionnaire (Table 2) was examined by three experts who had no involvement with the participants and were asked to rate each item. The final survey included only items with strong relevance, which was defined in our previous papers [20,21,22]. The questionnaire had a content validity index of 0.90.

Reliability

The internal consistency of all indexes was estimated using Cronbach’s α [23,24,25]. The questionnaire had a Cronbach’s coefficient α of 0.801.

Statistical analysis

The sample size analysis was calculated using power analysis and sample size (G-Power) software with at least an effective sample size with 80 % power at the 5 % significance level (2-sided test) in each group for EBP, with the hypothesis of no difference between the two groups. Based on previous results (Chen CJ. Journal of Medical Education 2020) that showed an estimated standard deviation of 2.4 for the e-book group, respectively 2.5 for the PowerPoint group, a sample size of 16 teachers in each group would be able to detect a difference of 2.4 between the two groups. [21]. The evaluation of both EBM skills and teaching effects was analyzed as a continuous variable and expressed as mean ± standard error or mean difference. We adopted Student’s t-test to compare teachers’ evaluation of their EBM skills by Fresno test before the class, after the class and the students’ assessment of the teachers’ instruction. We adopted repeated measures ANCOVA to compare teachers’ evaluation of their EBM skills by Fresno test both before and after the class. The teachers were divided into sub-groups according to the length of their clinical experience: less than 5 years, 5–10 years, and more than 10 years of experience. A p-value < 0.05 was considered statistically significant. All statistical analyses were performed using Statistical Package for Social Science (SPSS, version 12) software.

Results

Participants

Two hundred teachers were assessed for eligibility, but 120 ultimately did not consent to the assessment due to being too busy with work. The 80 teachers who provided their consent were enrolled and randomly assigned to the e-book group (n = 40) or the traditional PowerPoint group (n = 40). Seven teachers were excluded for incomplete assessments. The final sample included 35 teachers in the e-book group and 38 in the PowerPoint group (Fig. 1). The basic demographic data of all the teachers are shown in Table 1.

Analysis of the EBM skills of teachers with the pre- and post-Modified Fresno test

Prior to the class, the responses of each group assessed by the Modified Fresno test did not differ. After the class, repeated measures ANCOVA have been used in the second part of Table 2: the difference between the post-test value and the pre-test value for every teacher was then used to compare the two groups. The Modified Fresno test showed significant different total scores in the 12 questions between the e-book and PowerPoint groups (88.4 ± 1.75 vs. 82.3 ± 2.04, 95 %CI: 0.710-11.389, effect size = 2.999, p = 0.027). In particular, the fourth question about chose a case to explain which kind of research design was used for the study type of the question and explaining that choice (P = 0.011), and the seventh question about the important results expressed in the articles found (P = 0.023) significantly different between the two groups. (Table 2) But, only the fourth question differed significantly between the two groups compared the post-test to pre-test (P = 0.024) (Table 2).

Teaching effects with feedback from students

Teaching effects were better in the e-book group than in the PowerPoint group according to feedback from students with regard to four items. Those items were as follows: “I am satisfied with this lesson” (mean difference 0.226, P = 0.015), “The teaching was of high quality” (mean difference 0.285, P = 0.004), “This was a good teaching method” (mean difference 0.225, P = 0.027), and “It aroused my interest in EBM” (mean difference 0.358, P < 0.001) (Fig. 2a).

Fig. 2
figure 2

The effectiveness of the teaching between the e-book group and the PowerPoint group measured by the students of (a) all teachers, (b) physicians with less than 5 years’ clinical experience, (c) physicians with 5–10 years’ clinical experience, and (d) physicians with more than 10 years’ clinical experience. * P < 0.05

Full size image

In the sub-group of physicians with less than 5 years’ experience, students’ feedback was better in “It aroused my interest in EBM” in the e-book group than in the PowerPoint group (difference 0.415, P = 0.007) (Fig. 2b).

In the physicians with 5–10 years’ experience, the students gave the e-book group higher ratings than the PowerPoint group for the following six items: “I am satisfied with this lesson” (difference 0.601, P < 0.001), “The teaching was of high quality” (difference 0.593, P = 0.001), “The teacher had a good attitude” (difference 0.390, P = 0.012), “This was a good teaching method” (difference 0.869, P < 0.001), “It helped my understanding” (difference 0.407, P = 0.012), and “It aroused my interest in EBM” (difference 0.569, P < 0.001) (Fig. 2c).

However, we observed no difference in students’ feedback for the physicians with more than 10 years’ experience between the two groups (Fig. 2d).

Discussion

Improving teaching knowledge and problem-solving skills is important for teachers. Our study showed that the interactive e-book for EBM learning had better learning effects than those in the PowerPoint group according to the Modified Fresno test with regard to choose a case to explain which kind of research design was used for the study type of the question and explaining that choice. The interactive e-book learning improved the effectiveness of teaching in terms of greater satisfaction, better teaching quality, better teaching methods, and arousing students’ interest in EBM when compared to the traditional PowerPoint method. However, the groups with teachers who had less than 5 years’ experience and those with more than 10 years’ experience demonstrated fewer teaching differences.

The cause of the different learning and teaching effects regarding teachers’ seniority is not clear in this study, but may be related to the Taiwan Evidence-Based Medicine Association being established in 2007 [26]. We suppose that the different learning and teaching effects in the teachers’ seniority is the result of the promotion of EBM in Taiwan. One possible reason that physicians with 5–10 years’ experience are better at teaching is that the e-learning tutorial can provide an efficient and effective means of information delivery to junior doctors [27]. Therefore, they can share their experience of learning EBM, making it more interesting to their students.

The most commonly ranked barriers to the application of EBP include insufficient time, lack of skill in literature searching, and lack of skill in critical appraisal [28, 29]. In our study, the e-book group did not experience the problem of time, which offered significant improvement to the teachers because they could use it whenever convenient. The low rate of participation in our study may be due to physicians being too busy with work, and this may be improved for assessment on non-working days.

One previous study reported that approximately 30 % of residents did not complete their EBM exercises [30]. This finding indicates that a better teaching method is needed to improve teachers’ skills for students’ learning. Our interactive multimedia Internet e-book seems to meet this requirement. E-learning with structured materials provides predefined problem sets and has better student learning results [14, 31]. An e-learning approach to educating students to meet the criteria for evidence-based practice can result in higher-quality search strategies and improve confidence in EBM skills [32, 33]. The functions available in our e-book, combined with interactive quizzes and animations, resulted in improved learning effects in certain EBM skills.

In some cases, e-learning that uses e-books may have such challenges as cost and skill training [15], but our free Internet e-book does not have such problems. We used SimMAGIC to edit the e-book, which is simple and has a clear format. The cost of this software is inexpensive, and learners can use this e-book for free on the Internet [8]. Skills training in our e-book is not difficult, and as Persky has reported, an e-book is convenient in that it allows learners to pace themselves and have an active role in their learning [31]. Clear evidence from our data indicates that the teachers, especially those with 5–10 years’ experience, were able to extrapolate some of the knowledge gained from this practice.

Conclusions

The use of interactive e-books in clinical teaching can enhance a teacher’s EBM skills, especially those with 5–10 years of clinical experience, though not in more senior physicians. This may suggest that teaching methodology and activities differ for teachers’ varying years of experience.

Limitations

This study has certain limitations that are worth noting. Since it is a single-site study that uses a specifically designed instrument, our findings need to be tested in a large-scale study for generalizability to other populations in different locations. The low rate of participation in our study is another limitation that needs further consideration when interpreting its results.