Introduction

Basic science education has been a staple in medical education as students learn how to provide patient care. Retention of knowledge learned in these foundational courses declines over time [1]; however, it is necessary for knowledge to be retained beyond course assessments and throughout the preclinical and clinical years. Educational technologies may be able to aid in enhancing knowledge retention. Simulated clinical activities offer an endless variety of learning experiences for undergraduate medical students, which can aid in improving knowledge retention to bridge the gap between basic science education and clinical skills. The purpose of this study was to evaluate the outcomes of a simulation intervention on short-term student knowledge gain in first year medical students.

Literature Review

Retention of basic science knowledge during and beyond medical school has been a known challenge for well over a century, with the historical literature documented in a review by Custers [2]. He reports that if knowledge is to be retained, educators should review studies that show how to enhance long-term retention and include those strategies, such as creating repetitious practice sessions. Another strategy is to ensure students understand the relevance of what they are learning.

Chamberlain et al. [3] studied the impact of using case presentations to underscore the relevance of learning content from two basic science courses that would be needed later in clinical practice. Exam scores indicating knowledge retention were significantly higher for the students in the case-based group than those in the traditional lecture-laboratory setting. In a study to determine if medical students’ perceived relevance of biomedical sciences, demonstrated in clinical situations, would impact retention of basic science knowledge, Malau-Aduli et al. [4] found that higher perception of subject relevance correlated positively with increased retention of knowledge. They also suggest that using teaching strategies that increase awareness of clinical relevance will improve retention of knowledge.

Many preclinical medical educators, who formerly focused on verbal transmission of biomedical principles and factual information, are now presenting specific concepts with emphasis on clinical relevance [5]. Organ systems-based curricula increase integration and long-term retention of knowledge by presenting information organized around specific organ systems [2]. Following a longitudinal retrospective review after a curriculum revision, one study of medical students found that performance on the preclinical subject board examination, United States Medical Licensing Exam (USMLE) Step 1, and United States Medical Licensing Exam Step 2 was significantly better for those taught with the integrated organ systems curriculum than those taught with the traditional curriculum [6].

High-fidelity patient simulation is a teaching strategy that can combine case-based learning, developed within organ system-based curriculum, with a simulated clinical environment, allowing students to learn in context, thereby increasing the potential for students to see the relevancy of the material learned in the classroom that is now being applied to patient care experiences, and in a laboratory that can allow for repetition. The use of simulation, as a teaching strategy, offers an interactive means for organizing basic science knowledge around systems and disease processes, which may increase the perceived relevance of basic science education and thus increase knowledge retention.

The faculty developed a unique approach to teaching cardiovascular physiology using the Harvey® cardiovascular simulator in a basic science course [7]. We investigated whether the inclusion of a simulation event designed to allow first year medical students to apply basic science information learned in didactic lectures led to significant knowledge increases as assessed by course exam scores. With the growth of simulation use in medical education, it is important to investigate the impact of using simulation in undergraduate medical education and its role in a student’s ability to integrate, understand, and retain knowledge.

Methods and Materials

This investigation was approved by the Institutional Review Board of a large offshore United States (US) medical school where this study was conducted and subscribed to the tenets of the Declaration of Helsinki. Voluntary completion and return of the survey served as the informed consent to participate in the medical simulation research study. This study used historical controls for a comparison as curricular change had taken place. The medical school program admits three new classes of students per year (January, May, and September), which allows for the large sample sizes in this study. The control group that was chosen was the most recent group of students receiving the traditional curriculum (lecture only).

Sample

Two groups of students participated in this investigation. The first group was a historical control of 515 students from the 2009 matriculating class at the offshore US medical school. The group receiving the intervention was a group of 1066 students from the 2011 matriculating class at the medical school program at which the study took place. Twelve hours of cardiovascular physiology lectures were presented over 2 weeks, four of which were directly related to the simulation experience in this study. The day after the lectures ended, the students were divided into groups of eight to participate in a 1-hour (h) supplemental simulation exercise.

Instruction

Usual Instruction Method

Four hours of didactic physiology lectures directly related to the simulation were delivered during the first year cardiovascular course. The topics covered were the cardiac cycle (1 h), pressure-volume loops (1 h), and the physiological basis of heart sounds and murmurs (2 h). The content delivered was standard for any medical physiology course, based on the learning objectives provided by the American Physiological Society (APS) for these topics. The September 2009 and 2011 cohorts received the same lectures, delivered by the same faculty member, covering the same content.

Intervention

In addition to the didactic physiology lectures, the intervention group participated in a simulation exercise designed to reinforce the learning objectives of the didactic lectures. The Harvey Cardiovascular Simulator was used for simulation activities [7], and students completed clinical tasks including (1) stethoscope placement in the proper anatomical positions on the chest; (2) auscultation of first, second, third, and fourth heart sounds; and (3) auscultation of split heart sounds, with and without respiratory changes.

Clinically trained simulation facilitators attended the cardiovascular physiology lectures or reviewed lectures via video streaming to ensure a common understanding of what the students learned in lecture. Facilitators were given handouts that outlined the simulation objectives, content, and the desired instructional technique. A simulation training session was developed where facilitators discussed the objectives and content and a standardized approach to facilitation was demonstrated. After each cohort completed the simulation, there was a facilitator debrief where some videos of the simulations were reviewed and facilitators given feedback. This iterative process was repeated after each cohort completed the simulation.

Independent Variables

The independent variable assessed in this investigation was curriculum type—lecture-only curriculum versus lecture plus simulation.

Dependent Variables

Assessment of the cardiovascular course learning outcomes occurred via two examinations. The assessment questions were written in USMLE style and were identical for both cohorts. The first exam was a summative course exam at the end of the cardiovascular course; the second was a comprehensive final exam at the end of the semester that covered three consecutive courses. There were four assessment questions on the summative course exam and six on the comprehensive final exam that were used in this study, all directly mapped to the shared learning objectives for the physiology lectures and simulation exercise. The questions on the summative course exam and comprehensive final exam were different, but similar in content and difficulty. Scores were entered as percent correct. The cumulative average is the arithmetic average of the summative course exam score and comprehensive final score. The assessment was used to track differences in performance between the intervention group and the historical control group. The validity and reliability of the questions (and instruments) has not been assessed. Tests were administered through and data collected using Medical Education Technology, Incorporated’s, LearningSpace (METI LearningSpace®, now known as Canadian Aviation Electronics (CAE), [Limited] Healthcare’s LearningSpace) [8] a web-based clinical learning management system.

Analysis

Data were entered into a Microsoft Excel® [9] spreadsheet and saved into a comma-separated value (CSV) file. A frequency distribution of the teaching modality was calculated. Summative course exam grades, comprehensive final exam grades, and cumulative averages were tested for normality using normal probability plots and the Anderson-Darling, Shapiro-Francia, and the Shapiro-Wilk normality tests [10–12]. The Anderson-Darling test is the recommended empirical distribution function test by Stephens compared to other tests of normality giving more weight to the tails of the distribution than the Cramer-von Mises test [13]. The Shapiro-Francia test was chosen because of its known performance and the Shapiro-Wilk test was chosen because it is one of the best-known tests for normality [12]. An a priori alpha level for the goodness-of-fit tests was specified to be .10 [14]. Data were not normally distributed and thus the Wilcoxon rank sum (Mann-Whitney U) tests were used for significance testing. Since this is an educational intervention, an a priori alpha level of .10 was specified [15]. A two-sided test was used to test for differences in exam scores between curricula. All analyses were done using R software [16, 17].

Results

Five hundred fifteen (33 %) students participated in a lecture-only curriculum and 1066 (67 %) students participated in a curriculum augmented with simulation.

All exam scores (three) demonstrated marked departure from normality graphically and by all three tests of normality; therefore, hypothesis testing was done using the Wilcoxon rank sum (Mann-Whitney U) test. The central tendency and measures of dispersion, as well as p values, and associated confidence intervals can be seen in Table 1.

Table 1 Central tendency, p values, and 90 % confidence intervals for the means of three exam scores
Full size table

There were statistically significant differences between the knowledge outcomes for each teaching modalities for the summative course exam and the comprehensive final exam (p value = .0006 and <.0001, respectively). Not surprisingly, the average of these two exams, labeled “cumulative average of exam,” were also significantly different (p value = .0980). Mean scores were also substantially different in the cases of summative course exam scores and comprehensive final exam scores differing by over five points. However, these means were not consistently higher or lower. Summative course exam scores were higher for the simulation-plus-lecture group and comprehensive final exam scores were higher for the lecture-only group. Median scores for both exams were the same for both groups, 75 and 100 for the summative course exam and comprehensive final exam, respectively. There was no practical difference in averages of the two exams between the groups (one percentage point); however, there was a statistical difference owing to the large sample size.

In conclusion, mean summative course exam scores and mean comprehensive final exam scores were significantly and meaningfully different, but median scores were identical. In the case of the average of the two scores, mean scores were almost identical, but there was a five percentage point difference in median exam scores favoring the lecture curriculum.

Discussions

Results of this investigation demonstrated that first year medical students in a basic science curriculum augmented with simulation scored significantly better than historical controls on the summative course exam. According to Bowe et al. [5], preclinical educators now present concepts within a clinical relevance framework, which advances active learning, critical thinking, and other professional competencies rather than using the historical method of transmitting biomedical principles and facts via lecture. This investigation tested and reported the results of using simulation as a teaching strategy; our data analyzes the hypothesis that first year medical students learned cardiovascular physiology concepts better when lectures were paired with simulation when tested early in the term. Students also gained clinical skills earlier in the curriculum, as traditionally, the Harvey simulation was not introduced until the fourth semester midway through the second preclinical year at this medical school. One important benefit of using the Harvey with first semester students is that students were introduced to clinical skills in the first semester and provided clinical context of basic science education immediately.

Pairing simulation with cardiovascular physiology lectures may allow students to apply basic science knowledge earlier and within clinical context, and the corresponding relationship may narrow the retention gap traditionally seen in undergraduate medical education. A systemic review by Issenberg et al. [18] concluded that simulation-based education was effective and complements medical education. Increasing short-term retention, improving the retention interval, and transferring knowledge to practice is challenging because it takes more time to offer hands-on learning opportunities than it does to lecture. McGaghie et al. [19] call for integration throughout the curriculum to solve this challenge.

Students participated in the Harvey simulation sessions as first year students and again as second year students, doubling their simulation exposure, thus having had more opportunity to enhance their basic science knowledge and gain additional clinical skills. Simulation coupled with lectures in cardiovascular physiology allows students to learn through experience, and according to the course evaluations, it is valued highly by students. Using this paradigm, new knowledge is acquired in a clinical context, aiding subsequent retrieval and reinforcing current knowledge [1].

It was surprising to discover that there was not a significant difference in the comprehensive final exam grades between the two study groups. There may be several reasons for this lack of difference. One explanation could be attributed to the differences in intervals between time of exposure to the intervention and taking the summative course exam and the comprehensive final exam. The summative course exam took place approximately 2 weeks following the intervention while the comprehensive final took place a month later. During these periods, students did not have scheduled compulsory small group practice sessions with the Harvey that would allow for integrated review of the materials at least 1 week before their comprehensive final exam. Furthermore, the final exam was comprehensive; students are more likely to be overwhelmed when taking the comprehensive final since they are expected to study a larger volume of material involving multiple disciplines. In addition, they become anxious since the comprehensive final is a high-stakes exam accounting for a greater percentage weighting of their cumulative average. Students who fail the comprehensive final exam are at risk of having to remediate the entire program which may impose emotional distress due to adding four additional months to the time of the anticipated conferral of their doctorate of medicine degree, further contributing to their financial burden.

As with all studies, this study has its limitations. We did not assess the learner’s retention of the material in a longitudinal fashion. Additionally, no a priori power calculations were done; one could argue an adequate sample size was achieved. However, because of the larger sample size, a much smaller effect size was detectable, perhaps one not of importance. However, there were also meaningful differences between cardiovascular physiology scores of the historical cohort (78 % average) and the lecture-plus-simulation cohort (85 % average), which would result in the difference of a full letter grade for the student. Additionally, only facilitators and lecturers directly affiliated with the medical school were included in the study, and therefore, generalization must be made with caution. Finally, the simulation experience was facilitated by several different faculty members, which introduces variability in facilitation. To limit this variability, the facilitators underwent training of the simulation exercise together.

Future directions for investigation into knowledge gain would be to enhance participation. One possible approach would be to decrease the group size to better actively engage students in the learning activity. Another logical extension of this investigation would be to study longer term retention employing a longitudinal study within the term and in students’ second year of medical school using an experimental design approach. Finally, the number of questions should be increased on the assessment.

The findings in this study correlate to the historical literature [2] that established retention of content to be a challenge. While it was hypothesized that enhancing lecture with simulation would improve retention, in this study it did not. Placing the learning within context of a simulated environment using high-fidelity mannequins was found to improve scores initially similar to the findings of Chamberlain et al. [3], but not over time which further supports the need for repetition to lead to mastery. The findings of this study promote the need to ensure that opportunities to repeat active learning opportunities, such as those conducted in simulation environments, are made available and promoted for student practice.

Conclusions

The combination of simulation with the cardiac physiology lectures for first year medical students was found to have a significant impact on students’ cardiac physiology exam scores in the short-term. Offering additional practice sessions with the Harvey, throughout the term, could enhance knowledge retention of the information as repetition has been shown to positively impact retention. A longitudinal study is needed to see if there is long-term knowledge retention and improvement in clinical skills. Further studies are needed to replicate these findings in our student population regarding the magnitude of knowledge retention. Overall, the intervention was successful other than the limitations based on the small number of exam questions used to track students’ knowledge and the inherent variations among the trained facilitators. Thus, the quality of the outcomes depends primarily on faculty development and consistencies among facilitators.