In Part I of this two-part series, we examined the design and development of NERVE: A virtual patient simulation created to give medical students standardized experiences in interviewing, examining, and diagnosing virtual patients with cranial nerve disorders. We illustrated key design features and discussed how design-based research studies improved the total learning experience, including the virtual patient (VP) simulations and the instructional features incorporated with the simulations. In Part II, we examine the efficacy of NERVE and the strategy used to integrate the system into the medical school curriculum by field-testing it with 119 s-year medical students, and measuring students’ use, reactions, learning, and transfer. We report findings and reflect on lessons learned from the field-test to posit recommendations for improvement and guide the future research and development of virtual patient simulations.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
Ahmed, M., Sevdalis, N., Paige, J., Paragi-Gururaja, R., Nestel, D., & Arora, S. (2012). Association for Surgical Education: Identifying best practice guidelines for debriefing in surgery: a tri-continental study. The American Journal of Surgery, 203, 523–529.
Barrows, H. S. (1985). How to design a problem based curriculum for the preclinical years. New York: Springer.
Berman, N., Fall, L., Smith, S., Levine, D. A., Maloney, C. G., Potts, M., et al. (2009). Integration strategies for using virtual patients in clinical clerkships. Academic Medicine, 84(7), 943–949.
Botezatu, M., Hult, H., Kassaye Tessma, M., & Fors, U. G. (2010). As time goes by: Stakeholder opinions on the implementation and use of a virtual patient simulation system. Medical Teacher, 32(11), e509–e516.
Cendan, J., & Lok, B. (2012). The use of virtual patients in medical school curricula. Advances in Physiology Education, 36(1), 48–53.
Clark, R. E. (2004). Design Document for a Guided Experiential Learning Course. Submitted to satisfy contract DAAD 19-99-D-0046-0004 from TRADOC to the Institute for Creative Technologies and the Rossier School of Education, University of Southern California.
Consorti, F., Mancuso, R., Nocioni, M., & Piccolo, A. (2012). Efficacy of virtual patients in medical education: A meta-analysis of randomized studies. Computers & Education, 59(3), 1001–1008.
Cook, D. A. (2014). How much evidence does it take? A cumulative meta-analysis of outcomes of simulation-based education. Medical Education, 48(8), 750–760.
Cook, D. A., Erwin, P. J., & Triola, M. M. (2010). Computerized virtual patients in health professions education: A systematic review and meta-analysis. Academic Medicine, 85, 1589–1602.
Cook, D. A., & Triola, M. M. (2009). Virtual patients: a critical literature review and proposed next steps. Medical Education, 43(4), 303–311.
Dewey, J. (1938). Logic: The theory of inquiry. New York: Holt, Rinehart and Winston.
Edelbring, S. (2010). A three-fold framework for relating to innovations and technology in education: learning from, with and about technology. In A. Bromage, L. Clouder, J. Thistlethwaite, & F. Gordon (Eds.), Interprofessional E-learning and collaborative work: Practices and technologies (pp. 23–33). Hershey: IGI Global.
Edelbring, S., Broström, O., Henriksson, P., Vassiliou, D., Spaak, J., Dahlgren, L. O., & Zary, N. (2012). Integrating virtual patients into courses: follow-up seminars and perceived benefit. Medical Education, 46(4), 417–425.
Edelbring, S., Dastmalchi, M., Hult, H., Lundberg, I. E., & Dahlgren, L. O. (2011). Experiencing virtual patients in clinical learning: a phenomenological study. Advances in Health Sciences Education, 16(3), 331–345.
Fischer, M., Hege, I., Hörnlein, A., Puppe, F., Tönshoff, B., & Huwendiek, S. (2007). Virtual patients in medical education: a comparison of various strategies for curricular integration. Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen, 102(10), 648–653.
Haag, M., Singer, R., Bauch, M., Heid, J., Hess, F., & Leven, F. (2007). Challenges and perspectives of computer-assisted instruction in medical education. Lessons learned from seven years of experience with the CAMPUS system. Methods of Information in Medicine, 46(1), 67.
Halan, S., Rossen, B., Cendan, J., & Lok, B. (2010). High score! - motivation strategies for user participation in virtual human development. Intelligent Virtual, 9783642158919, 482. doi:10.1007/978-3-642-15892-6_52.
Hirumi, A., Johnson, T. & Reyes, R. J. (2015). Field-testing Strategies to Improve the Integration of Virtual Patient Simulations into Medical School Curriculum. Concurrent session presented the annual Association for Educational Communication and Technology conference, Indianapolis, IN. November 4–6.
Hirumi, A., Kleinsmith, A., Johnsen, K., Kubovec, S., Eakins, M., Bogert, K., et al. (2016). Advancing virtual patient simulations through design research and interplay: Part I: Design and development. Educational Technology Research and Development,. doi:10.1007/s11423-016-9429-6.
Huwendiek, S., & De Leng, B. A. (2010). Virtual patient design and curricular integration evaluation toolkit. Medical Education, 44(5), 519.
Huwendiek, S., Duncker, C., Reichert, F., De Leng, B. A., Dolmans, D., van der Vleuten, C. P., & Tönshoff, B. (2013). Learner preferences regarding integrating, sequencing and aligning virtual patients with other activities in the undergraduate medical curriculum: A focus group study. Medical Teacher, 35(11), 920–929.
Huwendiek, S., Reichert, F., Bosse, H. M., De Leng, B. A., van der Vleuten, C. P., Haag, M., & Tönshoff, B. (2009). Design principles for virtual patients: a focus group study among students. Medical Education, 43(6), 580–588.
Issenberg, S. B., McGaghie, W. C., Petrusa, E., Gordon, D. L., & Scalese, R. J. (2005). Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Medical Teacher, 27(1), 10–28.
Keller, J. M. (1987). Development and use of the ARCS model of instructional design. Journal of Instructional Development, 10(3), 2–10.
Keller, J. M. (2010). Motivation design for learning and performance: The ARCS model approach. New York: Springer.
Kirkpatrick, D. L. (1994). Evaluating training programs: The four levels. San Francisco: Berrett Koehler Publishers.
Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs: Prentice-Hall Inc.
Levett-Jones, T., & Lapkin, S. (2014). The effectiveness of debriefing in simulation-based learning for health professionals: A systematic review (Protocol). JBI Database of Systematic Reviews and Implementation Reports, 9, S81–S96.
Lindsey, L., & Berger, N. (2009). Experiential approach to instruction. In C. Reigeluth & A. Carr-Chellman (Eds.), Instructional-design theories and models (Vol. 3, pp. 117–142)., Building a common knowledge based New York: Routledge.
Maran, N. J., & Glavin, R. J. (2003). Low-to high-fidelity simulation – a continuum of medical education? Medical Education, 37(1), 22–28.
McGaghie, W., Issenberg, S., Cohen, E., Barsuk, J., & Wayne, D. (2011). Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Academic Medicine, 86(6), 706–711.
McGaghie, W. C., Issenberg, S. B., Petrusa, E. R., & Scalese, R. J. (2010). A critical review of simulation-based medical education research: 2003–2009. Medical Educator, 44, 50–63.
Paige, J. T., Arora, S., Fernandez, G., & Seymour, N. (2015). Debriefing 101: Training faculty to promote learning in simulation-based training. The American Journal of Surgery, 209(1), 126–131.
Pfeiffer, J. W., & Jones, J. E. (1975). Introduction to the structured experiences section. In J. E. Jones & J. W. Pfeiffer (Eds.), The 1975 annual handbook for group facilitators. La Jolla: University Associates.
Salem-Schatz, S., Ordin, D., & Mittman, B. (2010). Guide to the after action review v1.1. Accessed 27 September 2014 from http://as.vanderbilt.edu/overview/faculty/facultycouncil/archive/sitemason.vanderbilt.edu/files/cHpJCw/Guide%20to%20the%20After%20Action%20Review.pdf
Schank, R. C., Berman, T. R., & Macpherson, K. A. (1999). Learning by doing. In C. M. Reigeluth (Ed.), Instructional design theories and models: A new paradigm of instructional theory (pp. 161–179). Hillsdale: Lawrence Erlbaum Associates.
Stapleton, C., & Hirumi, A. (2011). Interplay instructional strategy: Learning by engaging interactive entertainment conventions. In M. Shaughnessy & S. Fulgham (Eds.), Pedagogical models: The discipline of online teaching (pp. 183–211). Hauppauge: Nova Science Publishers Inc.
Stapleton, C., & Hirumi, A. (2014). Designing InterPLAY learning landscapes to evoke emotions, spark the imagination, and promote creative problem solving. In A. Hirumi (Ed.), Grounded designs for online and hybrid learning: Practical guidelines for educators and instructional designers (pp. 159–190). Eugene: International Society for Technology in Education.
Research reported in this paper was supported by the National Institutes of Health (NIH) under award number 1R01LM010813-01.
Conflict of interest
The authors declare that they have no conflict of interest.
The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Rights and permissions
About this article
Cite this article
Hirumi, A., Johnson, T., Reyes, R.J. et al. Advancing virtual patient simulations through design research and interPLAY: part II—integration and field test. Education Tech Research Dev 64, 1301–1335 (2016). https://doi.org/10.1007/s11423-016-9461-6
- Virtual patient simulations
- Instructional design
- Design-based research