Development of high-fidelity patient simulators as a learning modality in the training of health professional students has been reviewed by Good [1], Issenberg et al. [2], Okuda et al. [3], and Gordon et al. [4]. Among the numerous benefits described are enhancement of assimilation/integration of basic science with clinical knowledge, acquisition of clinical procedure skills, student recognition of knowledge gaps and limitations, and assessment of teamwork and communication skills in small groups—all accomplished without putting actual patients at risk. Supplementing the cognitive aspects of this learning modality are emotive aspects, effected by hands-on patient interaction, that have been shown to enhance memory and recall [4, 5].

Much simulation to date has involved advanced-level medical students and residents. Simulation in training first-year medical students, who typically possess few clinical skills, generally has been coupled to basic science courses to reinforce difficult concepts, primarily in cardiac and respiratory physiology [6, 7] and neuroscience [8]. We wanted to pilot a simulation exercise linked with, and conducted similar to, our first-year problem-based learning (PBL) experience.

The qualitative research study reported herein can be considered a type of participatory action research that involves “planning a change, acting and observing the process and consequences of the change, reflecting on these processes and consequences, and then re-planning” and repeating the cycle [9]. In our case, the change involved adding simulation exercises to PBL. The research cycles included six simulation exercises (two repeated) over 3 years, with three classes of medical students, 115 students each. Several types of formative assessments [10] were used to evaluate the exercises, identify problems, re-plan them with improvements, and re-evaluate. The end goal was to develop successful simulation exercises that first-year medical students and non-physician PBL facilitators could actively participate in to enhance their PBL experience while providing them the opportunity to try out early clinical skills as they familiarize themselves with a patient simulation arena.

Simulation Coupled to Problem-Based Learning

PBL for first-year medical students at West Virginia University (WVU) is held 1½ hours/week following the traditional PBL format [11–13] and stresses the learning and reinforcement of basic science as it relates to understanding manifestations of clinical problems. Knowledge acquisition, teamwork, and communication are also assessed. There are no PBL content exams (performance is assessed through a written narrative), which helps to generate a less stressful atmosphere in which students can freely think and explore. Our ~40 faculty facilitators are from the basic science and various other educational departments, with only one MD and one MD/PhD in the mix, promoting the emphasis on basic science. Students uniformly comment about the more relaxed nature of this course, their enjoyment of its clinical relevance, and its reinforcement of their first-year courses.

Adding Patient Simulation to First-Year PBL

Current trends in medical education include a commitment to creating clinical experiences for students from the start. WVU has long embraced this model. Besides exposure via PBL cases, students also take their first-year, clinically based PDCI course (Physical Diagnosis and Clinical Integration). A new clinical teaching modality presented itself in 2009 when the School of Medicine established a high-fidelity patient simulation center, WV Simulation Training & Education for Patient Safety (STEPS), hereafter called the Sim Center. With encouragement from our Sim Center staff, and because certain PBL cases lend themselves well to simulation, we decided to pilot linking the simulation experience to PBL to expose our first-year students more realistically to the clinical aspects of a case. We sensed that this early simulation exposure would complement the concurrent instruction on taking a patient history and performing a physical exam, and would also prepare students for subsequent simulation encounters.

Student-Run Simulation as an Introduction to the PBL Patient

Current reports of coupling simulation to PBL for first-year medical and nursing students describe working through a PBL process initially, followed by a simulation exercise to bring that PBL patient to life [14–17]. (In another report, the progressive PBL paper sessions were actually replaced by simulation [18].) We decided to approach this in a different way, using a patient simulation exercise to introduce the PBL patient to the PBL group prior to their working through the paper case, thereby enhancing the subsequent PBL process. No diagnosis was expected, but rather an initial assessment and stabilization of the patient.

Another significant contrast of our PBL simulation exercise is that the students, rather than physicians, would run the exercise with the group’s PBL facilitator acting as a guide, similar to their roles in the PBL group process itself. Most facilitators had been exposed to these PBL cases several times, and they were given a hands-on training session for the associated simulation exercise (below). If a question arose that the facilitator could not address, the medically trained Sim Operator could intervene.

Challenges to Address

Five major challenges presented themselves in designing and executing patient simulation for first-year medical students in PBL. These are described in Table 1, along with the means employed to address them.

Table 1 Challenges in developing PBL simulation exercises for first-year medical students
Full size table

Choosing PBL Cases for Simulation

For the first simulation, we sought a PBL patient with clear-cut presenting symptoms and vital and physical signs that showed an unstable condition related to current basic science course material (cardiac physiology). Students were expected to take a patient history, interpret vital sign data from the monitor, and carry out a partial physical exam on the manikin. Chest X-rays were made accessible online in each room. The team would then make an evidence-based decision on treatment to stabilize the patient, with required items (e.g., oxygen mask) placed in the room for them to use and evaluate. They would finish by hypothesizing the next step(s) in this patient’s care, subsequently revealed through the normal three-session PBL process that followed.

To date, four PBL simulation exercises have been developed in alternating Fall/Spring pairs (Table 2). The first two exercises were re-used in Fall 2012 and Spring 2013, incorporating lessons learned.

Table 2 PBL cases used for patient simulation exercises
Full size table

Guidance Sheets for Students, Tutors and Simulation Operators

Student Guidance Sheets for each simulation exercise consisted of Goals, Patient presentation (initial stem of their PBL case), Stepwise objectives, and Wrap-up. A Patient Chart was provided, to be filled out as the exercise progressed. Additional information was provided in designing the Tutor Guidance Sheets to assist facilitation, especially by non-physicians. Operator Guidance Sheets contained the case stem and provided patient/family history, current meds, vital signs, physical exam features, and treatment(s) with expected results. From this information, the manikin could be made to “come to life” through the operator’s actions and respond appropriately to the situation at hand, including questions asked the “patient” by students and treatments provided. Items that needed to be in the room were listed, limiting exposed supplies to minimize student distraction and error. Close communication with our skilled operators was key to success.

Facilitator Training and Student Preparation for Pilot Simulations

PBL facilitators had access to online training materials and hands-on training sessions, with the operators and PBL course director/simulation designer explaining and showing what was involved. In the first year, the PBL facilitators were as unacquainted with simulation as the students, but their knowledge and comfort increased with experience.

The medical students had a brief tour of the Simulation Center during medical school orientation, and they were instructed to review online training materials prior to their simulation exercise. Students were given the Student Guidance Sheets and a 10-min overview of the exercise by their facilitator just before entering the Sim room. No further preparation was done the pilot year.

The fall PBL simulation exercise, held 2 weeks after students’ arrival at medical school, would be their first experience with the high-fidelity human manikins. Soon after, as part of the first-semester Human Function course (biochemistry/physiology), physiology faculty helped develop a simulation exercise in cardiac physiology, with a session on respiratory physiology added the following year. In view of the success of the Fall PBL simulation, a second PBL simulation exercise was developed for Spring.

Evaluation of the First Two Pilots

Four types of formative assessments were used to evaluate the new exercises. Immediately after the first PBL simulation, students were given two opportunities to provide feedback. The first was an individual online survey by the Sim Center, in which 45/115 (39 %) of students responded; 1/3 of these also provided written comments. The overwhelming response (relayed in 13 of 15 comments) was positive, with descriptors such as: “a great thing; worthwhile, eye opening; enjoyable, pertinent; useful/interesting; great program to help students get familiar with taking history and physical exams; made learning fun.” Some were reminded of how far they still needed to go towards working well with patients, while one student predicted that asking questions and examining the simulated patient would ease anxiety toward real patients. To the survey question “To what degree does the (Sim Center) provide an environment that fosters student learning?” the student rating was 4.38 out of 5.00. On the question “How strongly would you recommend that additional simulation exercises be incorporated into this course?” the rating was 4.13/5.00.

The second feedback opportunity involved each PBL group plus facilitator playing a “focus group” role in answering several open-ended questions (Appendix); the summarized comments are shown in Table 3. Consistent with the Sim Center survey, students found the simulation useful and informative for clinical skill development and integration of basic and clinical science. Uniformly, however, students felt that they were under-prepared for determining and conducting the patient care involved, since they were not familiar with clinical equipment and were unsure of what to do, though every group did complete each exercise with facilitator and operator guidance.

Table 3 Evaluations by PBL groups of the first two pilot simulation exercises
Full size table

In a third formative assessment, our two PBL facilitator physicians were interviewed by the PBL Course Director, and they provided several suggestions:

  • Clarify the progression of four steps that are to be accomplished:

    1. 1.

      Assess how/why patient is unstable through “vital” signs and initial history.

    2. 2.

      Stabilize patient/know how to assess when this is achieved (Keep it simple!).

    3. 3.

      Finish limited physical exam and history.

    4. 4.

      Discuss further tests and treatment to end the exercise.

  • Prior to the exercise, more instruction should be provided up front on what to do.

  • Students should be assured that the session is not trying to emphasize their lack of clinical knowledge, so they need not worry about feelings of inadequacy.

With student and facilitator consent, each simulation session was videotaped; a review of these by the PBL Course Director and a simulation Operator, as well as direct observation of sessions through the Sim Center’s one-way mirrors, completed the formative assessments and confirmed a number of feedback comments. They also revealed that students with medical experience can have an impact on a “team” effort. In one group, a student took over to the point that his peers backed away from the bed as he gave orders (some not understood) and performed, himself, the tasks required. In two other groups, students known to have medical expertise were much more subtle as they adjusted oxygen pressure, inserted an IV into an established port, and caught a key point in the patient history as their peers remained close around the bedside and fully involved.

Improvements Made in Year 2

Four major changes were made in the PBL simulations for the following year’s class. First, student preparation was supplemented the week before with a “Preliminary Student Copy” of the exercise (minus patient presentation, identifiers, and data) so that they could familiarize themselves with what they would be doing. In the trauma case, a brief tutorial was appended with explanations of lung sounds, vital signs, and the ATLS (Advanced Trauma Life Support) protocol’s Primary Survey, and students were warned that they would be going much more slowly and deliberately than in a real trauma situation.

Second, to help relieve some of the inadequacy students felt about patient care, advice about expectations was printed in color at the top of the preliminary document: “… As beginning MSI’s, you are NOT at all expected to be experts! It is OK to be unsure, and to ask questions. Your facilitator is prepared to help, as are WV STEPS operators. Relax, work with each other, explore, and enjoy the learning …” They were also advised on how to behave if they do have medical expertise and were reminded to view the online Sim Center information in advance to help them perform the tasks now described.

Third, during the 10-min orientation, facilitators had group members volunteer in pairs to perform four roles (patient history, physical exam, treatment, and recording), with all students’ eventually performing the physical exam. This helped to avoid the disorder seen when all seven to eight students tried to do everything. Listening to “normal” heart and lung sounds on a male student volunteer failed; subsequently, online recordings were made available in each Sim room.

Finally, more guidance was given on procedures. When students chose to administer oxygen, they were given a brief handout describing how to set up a non-rebreather mask. During the accident trauma victim case, our medically trained simulation operators demonstrated how to insert a chest tube after students discovered a pneumothorax from physical signs and chest X-ray. Students, facilitators, and operators enjoyed and benefited from this, and we now incorporate “operator teaching moments” into all the exercises (Table 2).

Evaluation of Year 2’s simulation exercises

Group evaluation forms were completed for the new Simulation exercises done by the following first-year class, with an 80 % return rate in Fall 2011 and 69 % return in Spring 2012. Enjoyment and interest remained high for many of the same reasons given in Table 3 by the previous year’s class and also because the exercise is not graded. Every group found the operator explanation/demonstration of the chest tube insertion especially useful. Although still admitting discomfort with ignorance in clinical/trauma skills, this was mentioned less while feeling prepared was mentioned more, as was achieving the goals and objectives. With the assigning of roles to pairs of students, there was less mention of disorganization. Upon viewing the session videotapes, the PBL Course Director and a Sim Center Operator observed less disorder and discomfort.

Year 3: First Repetition

In Fall 2012 and Spring 2013, the first two simulation pilots were repeated, with improvements. Evaluations from 80 % of Fall groups and 62 % of Spring groups were received. Ninety percent reported achieving objectives, with no negative comments concerning clarity. A variety of “most informative parts” were mentioned besides those from Table 3: learning how to give oxygen (the well-received operator teaching moment), consideration for patient modesty, that taking vital signs is vital (!), exposure to the ABC’s of trauma care, having to think on their feet and respond, and the helpful and slowed-down step-by-step trauma care. Most groups now felt prepared for the exercises, and no mention was made of group disorder. Groups continued to comment, however, on difficulties listening to and understanding heart and lung sounds, an area for future improvement.

Discussion

Qualitative research seeks to understand people’s interpretations of natural situations in which they are involved [19, 20]. As we describe in this report, one type of question suited to qualitative inquiry involves the quality or success of a particular innovation or program [9, 21]. The collaboration of teachers and students to make improvements in teaching and learning in the classroom exemplifies “participatory action” qualitative research [9]. The end goal of our study was to develop successful PBL-linked simulation exercises that would actively engage first-year medical students to enhance their PBL experience while giving them the opportunity to try out clinical skills as they familiarized themselves with the patient simulation arena. The research cycles (planning, acting/observing, reflecting, and re-planning) involved six simulation exercises (two repeated) over 3 years, with three classes of medical students, 115 students each. Our goal was largely achieved through this process that allowed us to identify and implement strategies for improvement as the exercises were developed.

Qualitative research data can be collected through direct observation of behaviors, as well as indirect means, such as questionnaires. We employed both types in our simulation exercises. The simulation sessions and videotape reviews employed direct observation, while the focus group (PBL group) surveys and physician facilitator interviews were indirect.

Unlike quantitative research in which the investigator is generally removed from the investigation itself, participation of the qualitative researcher can vary from external or passive to being a total natural participant [20]. In our research, the authors/observers were PBL facilitators, Sim Center Operators, and a medical PBL student, thus were total natural participants during the actual simulation sessions and the physician interviews, closely interacting with the research subjects. At other points of investigation, the authors were external or passive participants, such as when viewing sessions from behind the one-way mirrors or on videotape, and when conducting the formative assessment group surveys. Along with these multiple observation perspectives, the authors’ decades of experience teaching and interacting with medical students, and their intimate and broad knowledge of the process being studied, enabled them to gain a deeper understanding of the motivations, behaviors, and meanings of the research subjects, and importantly, make appropriate changes for improvement.

Corroboration in qualitative research helps ensure that the findings accurately reflect the perceptions of those observed (and not whether those perceptions are right or wrong). Triangulation (or crystallization) is one means of corroboration used to achieve an in-depth understanding of the phenomenon in question [19–22]. In our study, this involved not only the convergence of the formative data from multiple sources (see below), but also multiple researchers/observers with diverse roles and backgrounds providing input and interpreting the formative assessments. This multi-method research strategy is reported to add “rigor, breadth, complexity, richness and depth” to any qualitative inquiry [22].

Adding a high-fidelity simulation exercise to introduce first-year medical students to a PBL patient has a number of benefits for problem-based learning. Confirming previous reports on simulation in general [1–4], it enhances the learning and integration of basic and clinical science, as our PBL students explain the reasons for observed clinical symptoms and justify their treatment choices. Simulation helps students acquire clinical procedure skills, in our cases, by conducting an appropriate patient assessment and administering stabilizing treatments, enhanced by the operator’s “teaching moments.” The group simulation exercises foster teamwork and communication skills, as students have to work together to assess the patient and determine what their findings mean and how to proceed. Finally, the formative assessments from the students (and novice facilitators) provide evidence that these exercises are valuable metacognitive experiences [23, 24], allowing participants to recognize knowledge gaps and limitations, especially relating to clinical skills, and to begin to close those gaps.

Our simulation exercises are embedded within the PBL process as students take the introduction to the patient and build upon it as the case is progressively revealed and worked out on paper and white board. In this sense, the PBL exercise can be considered a “debriefing process” for the preceding simulation, an important component of experiential learning [3, 25]. In a successful exercise, the simulation fosters group-thinking, communication, teamwork, and camaraderie which are major goals of PBL. The bulk of students clearly enjoyed the exercises, which is one of the recognized positive aspects of problem-based learning itself compared with traditional lectures [26, 27], and they requested more simulation cases. In addition, positive effects were reported on the subsequent PBL sessions on the case:

  • Put a “face” on the case and gave a practical connection/helped visualize it/added texture,

  • Students experienced patient in real time with needs and personality,

  • It was great to work on the manikin first before getting case data/gave insight into upcoming case,

  • Physical interaction made later weeks of PBL more relevant and enjoyable,

  • Students got to ask their own questions,

  • Created a reference point as to how a physician might first interact with a patient/created more of a doctor–patient bond than the paper case/became more invested in patient’s welfare,

  • Made students think about the case more/made it more memorable/helped students remember what was learned/better understanding of test results and what was happening to the body, and

  • Things missed during the exercise were helpful for discussing the case in PBL.

Several of these impacts are consistent with those reported in other PBL simulation exercises [14–18]; however, our positioning of the simulation ahead of the PBL sessions was unique, changing the perspective and resulting in more direct benefits to the PBL process itself. In support of simulation’s enhancing learning and retention, Liaw et al. [17] reported that coupling simulation of a crisis event to PBL resulted in significantly higher test scores compared with PBL alone. Gordon et al. report that linking simulation to a myocardial infarction case study resulted in enhanced test performance in cardiac physiology that was sustained a year later [15].

Extensive feedback during the first-year pilots was gathered to determine what went well and identify areas for improvement by conducting formative assessment in four different ways. First, an online Sim Center survey of individual students was done after the first pilot. Although the responses showed a high level of consistency favoring the exercises, the poor response rate (39 %) created the possibility of sampling bias and led us to discontinue use of this instrument for future exercises. This rate is consistent with the 33 %, on average, found by Nulty [28] in summarizing eight studies comparing online surveys verses paper surveys administered face-to-face. Ways exist to improve this response rate [28], but we chose, instead, to employ other means of formative assessment.

Consistent with Nulty’s findings, significantly higher response rates were achieved when feedback was gathered by providing each group as a whole, with their facilitators, a paper copy of open-ended questions to be completed through group discussion during a subsequent PBL session. Furthermore, the constructive comments led to continued use of this instrument which is similar to the survey that we have long-used to evaluate each year’s PBL cases. The consistently higher group response rates in fall (73–80 %) compared with Spring (62–69 %) are also seen in our MSI student evaluations in general, and likely reflect, in part, the student mindset by the end of their first rigorous year of study.

Two additional means of formative assessment (interviewing the two physician facilitators and studying videotapes of each PBL group’s simulation session) added credence to the survey results and helped in altering the exercises to address problems and concerns. That the changes did effect improvement was documented through subsequent formative assessments the following years, using the same group survey questions for straightforward comparisons. The fact that most facilitators remained the same from year to year reduced that variability, and their increased familiarity with the simulation environment would be expected to contribute, at least in part, to their subsequent groups’ success.

In contrast to other reported studies of simulation linked to PBL, in which faculty or physicians generally directed the exercises [14–17], our simulations were student-driven, with the students in control, similar to their subsequent PBL sessions. In view of students’ general lack of medical experience, this format had the potential to elevate stress levels, a significant concern which we addressed in several ways, as described above. Furthermore, having non-expert (although trained) facilitators and no exams/grades were anticipated to lower the pressure on students to perform well. [PBL skills are evaluated by facilitator-written narratives, yet students are highly engaged, showing that exams are not required to motivate them to do well.] Although a number of student comments related to becoming more comfortable around patients and peers, a certain level of discomfort remained evident, even after enhancing student preparation. Coupled to their admitted feelings of inadequacy when it came to treating the “patient” is the documented emotional connection made between students and manikin [3–5], even though no actual patient is being put at risk. We observed this connection directly and in session videotapes. Despite these issues, the groups did accomplish the simulation objectives, typically by working together to figure out what to do, with some facilitator/operator guidance.

In his evaluation of the first-year pilots, our medical director for WV STEPS, co-author Dr. David Wilks (Anesthesiology), provided what might be the best perspective on this stress, consistent with the reported emotive benefits [4, 5] of learning: “Perhaps the learner should feel uncomfortable. They are at the very beginning of their medical learning experiences and indeed do not know how to take care of these patients. The purpose of the simulation is to give them a sense of why they are learning this material. When the case is presented totally on paper, the sense of urgency and the emotive aspects are not there. Emotion has been shown to be a very strong aspect of the learning process. This sense of discomfort may be a sign of success.”