The choice of outcome measure for simulation studies is a crucial element of the research design, for without careful forethought and planning how can we be confident that we are measuring what we intend to measure? In this chapter, we follow on from the concepts introduced in Chap. 25, outlining the key elements in developing and examining the validity argument for the outcome measure used in a simulation research study, with an emphasis on Kane’s framework.
KeywordsValidity Reliability Assessment Research design Simulation
- 3.Messick S. Validity. In: Linn RL, editor. Educational measurement. 3rd ed. New York: American Council on Education and Macmillan; 1989. p. 13–104.Google Scholar
- 5.Cook DA, Kuper A, Hatala R, Ginsburg S. When assessment data are words: validity evidence for qualitative educational assessments. Acad Med. 2016;91:1359–69. https://doi.org/10.1097/ACM.0000000000001175.
- 12.Cook DA, Zendejas B, Hamstra SJ, Hatala R, Brydges R. What counts as validity evidence? Examples and prevalence in a systematic review of simulation-based assessment. Adv Health Sci Educ Theory Pract. 2014;19(2):233–50.Google Scholar
- 14.Cook DA, Brydges R, Zendejas B, Hamstra SJ, Hatala R. Technology-enhanced simulation to assess health professionals: a systematic review of validity evidence, research methods, and reporting quality. Acad Med. 2013;88(6):872–83. https://doi.org/10.1097/ACM.0b013e31828ffdcf.
- 15.Hatala R, Sawatsky AP, Dudek N, Ginsburg S, Cook DA. Using In-Training Evaluation Report (ITER) qualitative comments to assess medical students and residents: a systematic review. Acad Med. 2017;92(6):868–79. https://doi.org/10.1097/ACM.0000000000001506.