This study compared the effects of two types of delayed feedback (correct response or correct response + rationale) provided to students by a computer-based testing system following an exam. The preclinical medical curriculum at the University of Kansas Medical Center uses a two-exam system for summative assessments in which students test, revisit material, and then re-test (same content, different questions), with the higher score used to determine the students’ grades. Using a quasi-experimental design and data collected during the normal course of instruction, test and re-test scores from midterm multiple choice examinations were compared between academic year (AY) 2015–2016, which provided delayed feedback with the correct answer only, and AY 2016–2017, where delayed feedback consisted of the correct answer plus a rationale. The average increase in score on the re-test was 2.29 ± 6.83% (n = 192) with correct answer only and 3.92 ± 7.12% (n = 197) with rationales (p < 0.05). The effect of the rationales was not different in students of differing academic abilities based on entering composite MCAT scores or Year 1 GPA. Thus, delayed feedback with exam question rationales resulted in a greater increase in exam score between the test and re-test than feedback with correct response only. This finding suggests that delayed elaborative feedback on a summative exam produced a small, but significant, improvement in learning, in medical students.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Balch, W. R. (1998). Pratice versus review exams and final exam performance. Teaching of Psychology, 25(3), 181–185.
Bangert-Drowns, R. L., Kulik, C.-L. L., Kulik, J. A., & Morgan, M. T. (1991). The instructional effect of feedback in test-like events. Review of Educational Research, 61(2), 213–238.
Butler, A. D., Karpicke, J. D., & Roediger, H. L., III. (2007). The effect and type and timing of feedback on learning from mulitple-choice tests. Journal of Experimental Psychology, 13(4), 273–281.
Clariana, R. B., Ross, S. M., & Morrison, G. R. (1991). The effects of different feedback strategies using computer-administered multiple-choice questions as instruction. Educational Technology Research and Development, 39(2), 5–17.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. New Jersey: Lawrence Erlbaum Associates.
DiBattista, D., Mitterrer, J. O., & Gosse, L. (2004). Acceptance by undergraduates of the immediate feedback assessment technique mfor mulitple-choic testing. Teaching in Higher Education, 9(1), 17–28.
Dihoff, R. A., Brosvic, G. M., & Epstein, M. L. (2003). The role of feedback during academic testing: The delay retention effect revisited. Psychological Record, 53(4), 533–548.
Hays, M. J., Kornell, N., & Bjork, R. A. (2010). The costs and benefits of providing feedback during learning. Psychonomic Bulletin & Review, 17(6), 797–801.
Kang, S. H. K., McDermott, K. G., & Roediger, H. L., III. (2007). Test format and corrective feedback modify the effect of testing on leng-term rentention. European Journal of Congnitive Psychology, 19(4/5), 528–558.
Krathwohl, D. R. (2002). A revision of Bloom’s Taxonomy: An overview. Theory Into Practice, 41(4), 212–218.
Kulhavy, R. W. (1977). Feedback in written instruction. Review of Educational Research, 47(1), 211–232.
Kulick, J. A., & Kulik, C. L. (1988). Timing of feedback and verbal learning. Review of Educational Research, 58(1), 79–97.
Levant, B., & Paolo, A. (2017). A between-exam review does not affect re-test performance of medical students in a two-test system. Medical Science Educator, 27, 29–32.
McDaniel, M. A., Roediger, H. L., III, & McDermott, K. G. (2007). Generalizing test-enhanced learning from the laboratory to the classroom. Psychonomic Bulletin & Review, 14(2), 200–206.
Merrill, J. (1987). Levels of questioning and forms of feedback: Instructional factors in courseware design. Journal of Computer-based Instruction, 14(1), 18–22.
Morrison, G. R., Ross, S. M., Gopalakrishnan, M., & Casey, J. (1995). The effects of feedback and incentives on achievement in computer-based instruction. Contemporary Educational Psychology, 20, 32–50.
Roediger, H. L., III, & Karpicke, J. D. (2006). Test-enhanced learning. Psychological Science, 17(3), 249–255. https://doi.org/10.1111/j.1467-9280.2006.01693.x.
Royal, K. D., Henderson, A. G., & Hedgpeth, M.-W. (2015). Post-exam reviews: A consideration of costs and uninteded consequences. Medical Science Educator, 25, 327–329.
Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.
Sinha, N., & Glass, A. L. (2015). Delayed, but not immediate, feedback after multiple-cnoice questions increases performance on subsequent short-answer, but not multiple-choice, exam: Evidence for the dual-porcess theory of memory. Journal of General Psychology, 142(2), 118–134.
Smith, T. A., & Kimball, D. R. (2010). Learning from feedback: Spacing and the delay-renetion effect. Journal of Experimental Psychology, 36(1), 80–95.
van der Kleij, F. M., Eggen, T. J. H. M., Timmers, C. F., & Veldkamp, B. P. (2012). Effects of feedback in a computer-based assessment for learning. Computers & Education, 58, 263–272.
Winniger, S. R. (2005). Using your tests to teach: Formative summative assessment. Teaching of Psychology, 32(3), 164–166.
The authors thank Tim Doughty for his assistance with ExamSoft.
Conflict of interest
The authors declare that they have no conflicts of interest.
Human and animal rights
All procedures performed in studies involving human participants were in accordance with the ethical standards of the University of Kansas Medical Center Institutional Review Board (Study #2981, approved 7/5/2017) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. The study is retrospective. For this type of study, formal consent is not required.
About this article
Cite this article
Levant, B., Zückert, W. & Paolo, A. Post-exam feedback with question rationales improves re-test performance of medical students on a multiple-choice exam. Adv in Health Sci Educ 23, 995–1003 (2018). https://doi.org/10.1007/s10459-018-9844-z
- Delayed feedback
- Elaborative feedback
- Medical student
- Multiple-choice question
- Computer-based testing
- Summative assessment