Advertisement

Advances in Health Sciences Education

, Volume 21, Issue 1, pp 221–234 | Cite as

Progress testing: critical analysis and suggested practices

  • Mark Albanese
  • Susan M. Case
Review

Abstract

Educators have long lamented the tendency of students to engage in rote memorization in preparation for tests rather than engaging in deep learning where they attempt to gain meaning from their studies. Rote memorization driven by objective exams has been termed a steering effect. Progress testing (PT), in which a comprehensive examination sampling all of medicine is administered repeatedly throughout the entire curriculum, was developed with the stated aim of breaking the steering effect of examinations and of promoting deep learning. PT is an approach historically linked to problem-based learning (PBL) although there is a growing recognition of its applicability more broadly. The purpose of this article is to summarize the salient features of PT drawn from the literature, provide a critical review of these features based upon the same literature and psychometric considerations drawn from the Standards for Educational and Psychological Testing and provide considerations of what should be part of best practices in applying PT from an evidence-based and a psychometric perspective.

Keywords

Progress testing Longitudinal assessment Formative assessment High stakes assessment 

References

  1. Albanese, M. A. (1986). The correction for guessing: A further analysis of Angoff and Schrader. Journal of Educational Measurement, 23(3), 225–235.CrossRefGoogle Scholar
  2. Albanese, M. (2008). Benchmarking progress tests for cross-institutional comparisons: Which road taken makes a difference and all roads have bumps. Medical Education, 42, 4–7.CrossRefGoogle Scholar
  3. Arnold, L., & Willoughby, T. L. (1990). The quarterly profile examination. Academic Medicine, 65, 515–516.CrossRefGoogle Scholar
  4. Blake, J. M., Norman, G. R., Keane, D. R., Mueller, C. B., Cunnington, J., & Didyk, N. (1996). Introducing progress testing in McMaster University’s problem-based medical curriculum: Psychometric properties and effect on learning. Academic Medicine, 71(9), 1002–1007.CrossRefGoogle Scholar
  5. Cross, L. H., & Frary, R. B. (1977). An empirical test of Lord’s theoretical results regarding formula scoring of multiple choice tests. Journal of Educational Measurement, 14(4), 313–321.CrossRefGoogle Scholar
  6. Joint Committee on Standards for Educational and Psychological Testing of the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington DC: American Educational Research Association.Google Scholar
  7. Kolen, M. J., (1988). Traditional equating methodology. Educational Measurement: Issues and Practice 115.Google Scholar
  8. Kolen, M. J., & Brennan, R. L. (1995). Test equating. New York: Springer.CrossRefGoogle Scholar
  9. Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121–1134.CrossRefGoogle Scholar
  10. McHarg, J., Bradley, P., Chamberlain, S., Ricketts, C., Searle, J., & McLachlan, J. C. (2005). Assessment of progress tests. Medical Education, 39, 221–227.CrossRefGoogle Scholar
  11. Muijtjens, A. M. M., Schuwirth, L. W. T., Coen-Schotanus, J., Thoben, A. J. N. M., & van der Vleuten, C. P. M. (2008). Benchmarking by cross-institutional comparison of student achievement in a progress test. Medical Education, 42, 82–88.CrossRefGoogle Scholar
  12. Muijtjens, A. M. M., van Mameren, H., Hoogenboom, R. J. I., Evers, J. L. H., & van der Vleuten, C. P. M. (1999). The effect of a ‘don’t know’ option on test scores: number-right and formula scoring compared. Medical Education, 33, 267–275.CrossRefGoogle Scholar
  13. Norman, G. R., Neville, A., Blake, J. M., & Mueller, B. (2010a). Assessment steers learning down the right road: Impact of progress testing on licensing examination performance. Medical Teacher, 32, 496–499.CrossRefGoogle Scholar
  14. Norman, G., Neville, A., Blake, J. M., & Mueller, B. (2010b). Assessment steers learning down the right road: Impact of progress testing on licensing examination performance. Medical Teacher, 32(6), 496–499.CrossRefGoogle Scholar
  15. Personal communication with Louis Arnold, August 31, 2007.Google Scholar
  16. Rademakers, J., Ten Cate, T. J., & Bär, P. R. (2005). Progress testing with short answer questions. Medical Teacher, 27(7), 578–582.CrossRefGoogle Scholar
  17. Swanson, D. B., Holtzman, K. Z., & Bulter, A. (2010). Cumulative achievement testing: Progress testing in reverse. Medical Teacher, 32(6), 516–520.CrossRefGoogle Scholar
  18. Traub, R. E., Hambleton, R. K., & Singh, B. (1969). Effects of promised reward and threatened penalty on performance of a multiple-choice vocabulary test. Educational and Psychological Measurement, 29, 847–861.CrossRefGoogle Scholar
  19. Van der Vleuten, C. P., Schuwirth, L. W., Muijtjens, A. M., Thoben, A. J., Cohen-Schotanus, J., & van Boven, C. P. (2004). Cross institutional collaboration in assessment: A case on progress testing. Medical Teacher, 26(8), 719–725.CrossRefGoogle Scholar
  20. Van der Vleuten, C. P. M., Verwijnen, G. M., & Wijnen, H. F. W. (1996). Fifteen years of experience with progress testing in a problem-based learning curriculum. Medical Teacher, 18, 102–109.CrossRefGoogle Scholar
  21. Verhoeven, B. H., Snellen-Balendong, H. A., Hay, I. T., Boon, J. M., van der Linde, M. J., Blitz-Lindeque, J. J., et al. (2005). The versatility of progress testing assessed in an international context: A start for benchmarking global standardization? Medical Teacher, 27(6), 514–520.CrossRefGoogle Scholar
  22. Verhoeven, B. H., Van der Steeg, A. F. W., Scherpbier, A. J. J. A., Muijtjens, A. M. M., Verwijnen, G. M., & van der Vleuten, C. P. M. (1999). Reliability and credibility of an Angoff standard setting procedure in progress testing using recent graduates as judges. Medical Education, 33, 832–837.CrossRefGoogle Scholar
  23. Willoughby, T. L., Dimond, E. G., & Smull, N. W. (1977). Correlation of quarterly profile examination and national board of medical examiner scores. Educational and Psychological Measurement, 37, 445–449.CrossRefGoogle Scholar
  24. Willoughby, T. L., & Hutcheson, S. J. (1978). Edumetric validity of the quarterly profile examination. Educational and Psychological Measurement, 38, 1057–1061.CrossRefGoogle Scholar
  25. Woolliscroft, J. O., Swanson, D. B., Case, S. M., & Ripkey, D. R. (1995). Monitoring the effectiveness of the clinical curriculum: Use of a cross-clerkship exam to assess development of diagnostic skills. In A. I. Rothman & R. Cohen (Eds.), Proceedings of the sixth Ottawa conference on medical education (pp. 476–478). Toronto: University of Toronto Bookstore Custom Publishing.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2015

Authors and Affiliations

  1. 1.University of Wisconsin-Madison Medical School, National Conference of Bar ExaminersMadisonUSA
  2. 2.AjijicMexico

Personalised recommendations