Examining the Effectiveness, Efficiency, and Usability of a Web-Based Experiential Role-Playing Aging Simulation Using Formative Assessment


We focused our research on the formative assessment of a web-based, role-playing simulation to better understand its effectiveness, efficiency, and usability prior to implementation. Participants received an email invitation with a link to the simulation. After completing the simulation activities, participants were asked to complete an electronic questionnaire. In addition, participants were offered incentives to participate in a face-to-face think-aloud protocol, in which the participants navigated the simulation while answering questions and describing their thoughts out loud to the researcher. In total, forty-two participants completed the electronic survey and ten participants completed the face-to-face think-aloud protocol. The results indicate that participants found that the web-based, role-playing simulation adequately synthesized fundamental knowledge and skills; it is organized in a way that helped the learner; and it is easy to navigate. However, this formative assessment of experiential role-playing aging simulation also showed that there are improvements and adjustments that should be considered before implementing it as an instructional tool. Although participants were eager to learn more about the subject presented, the message design (graphics, animations, and colors) needed careful consideration and revision.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 99

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5


  1. Adamson, K. A., & Kardong-Edgren, S. (2012). A method and resources for assessing the reliability of simulation evaluation instruments. Nursing Education Perspectives, 33(5), 334–339.

  2. Adcock, A. B., Duggan, M. H., & Perry, T. (2010). The evaluation of interactive learning modules to reinforce helping skills in a web-based interview simulation training environment. Educational Media International, 47(4), 293–310.

  3. Alessi, S. M., & Trollip, S. R. (2001). Multimedia for learning: methods and development (3rd ed.). Boston, MA: Allyn and Bacon.

  4. Buchanan, J. A. (2004). Experience with virtual reality-based technology in teaching restorative dental procedures. Journal of Dental Education, 68(12), 1 258–1 265.

  5. Chyung, S.Y. (2018). Ten-step evaluation for training and performance improvement: a systematic and systemic approach to designing and conducting program evaluation. Boise State University, SAGE.

  6. Creswell, J. W. (2009). Research design qualitative, quantitative, and mixed methods approaches (Third Edition ed.). Los Angeles, CA: SAGE publications.

  7. Kennedy, B. (2018). A UX introduction to instructional design. UserZoom. Retrieved from:

  8. Kiili, B. (2007). Foundation for problem-based gaming. British Journal of Educational Technology, 38(3), 394–404.

  9. Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. New York, NY: Prentice Hall.

  10. Logie, C., Bogo, M., Regehr, C., & Regehr, G. (2013). A critical appraisal of the use of standardized client simulations in social work education. Journal of Social Work Education, 49, 66–80.

  11. Moule, P. (2011). Simulation in nurse education: past, present and future. Nurse Education Today, 31, 645–646.

  12. Norman, D. (1988). The design of everyday things. New York, NY: Basic Books. 

  13. Palmore, E. B. (1998). The facts on aging quiz. New York: Springer Publishing Company.

  14. Patton, M. Q. (2015). Qualitative research and evaluation methods (fourth edition ed.). Thousand Oaks, CA: SAGE publications.

  15. Patrick, R., Romero-Hall, E., Iasillo, K., & Dwy, M. (2018). Promoting empathic concern for the elderly through a computer-based simulation model. In R. L. Miller & A. Martinez (Eds.). Teaching Tips: A compendium of conference presentations on teaching, 2016-17. Retrieved from the Society for the Teaching of Psychology web site: Accessed 27 Feb 2019.

  16. Plummer, K. (2001). The call of life stories in ethnographic research. In P. Atkinson, A. Coffey, S. Delamont, J. Lofland, & L. Lofland (Eds.), Handbook of ethnography. SAGE Publications.

  17. Pol, H., Harskamp, E., & Suhre, C. (2005). Solving physics problems with the help of computer-assisted instruction. International Journal of Science Education, 27(4), 451–469.

  18. Pol, H., Harskamp, E., Suhre, C., & Goedhart, M. (2008). The effect of hints and model answers in a student-controlled problem-solving program for secondary physics education. Journal of Science Education and Technology, 17(4), 410–425.

  19. Romero-Hall, E. (2016a). Evaluation of an instructional design and technology master level program. In Proceedings of EdMedia 2016: World conference on educational media and technology, pp. 1604–1611. Vancouver, BC, Canada: Association for the Advancement of Computing in Education (AACE).

  20. Romero-Hall, E. (2016b). A framework for designing instructional computer-based simulations: Research-based strategies for appropriate design. In Proceedings of EdMedia 2016--World conference on educational media and technology, pp. 1604–1611. Vancouver, BC, Canada: Association for the Advancement of Computing in Education (AACE).

  21. Romero-Hall, E. J., Patrick, R., & Sahin, G. (2019). ERAS: An experiential role-playing aging simulation. International Journal of Designs for Learning, 10(1), 14–24.

  22. Smith, M. (2009). Creative clinical solutions: aligning simulation with authentic clinical experiences. Nursing Education Perspectives, 30(2), 126–128.

  23. Steinberg, G. (2012). Natural user interfaces. Paper presented at the ACM SIGCHI conference on human factors in computing systems, Austin, TX.

  24. Sullivan-Mann, J., Perron, C. A., & Fellner, A. N. (2009). The effects of simulation on nursing students’ critical thinking scores: a quantitative study. Newborn & Infant Nursing Reviews, 9, 111–116.

  25. Tessmer, M. (1993). Planning and conducting formative evaluations. New York, NY: Routledge.

  26. Wolgin, M., Wiedemann, P., Frank, W., Wrbas, K., & Kielbassa, A. M. (2005). Development and evaluation of an endodontic simulation model for dental students. Journal of Dental Education, 79(11), 1363–1372.

  27. Yin, R. (2014). Case study research: design and methods (applied social research methods). Thousand Oaks, CA: SAGE Publications.

Download references

Author information

Correspondence to Enilda Romero-Hall.

Ethics declarations

Conflict of Interest

The authors declare that there they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Romero-Hall, E., Adams, L. & Osgood, M. Examining the Effectiveness, Efficiency, and Usability of a Web-Based Experiential Role-Playing Aging Simulation Using Formative Assessment. J Form Des Learn 3, 123–132 (2019).

Download citation


  • Formative assessment
  • Instructional design
  • Simulations
  • Games