Advertisement

Advances in Health Sciences Education

, Volume 11, Issue 2, pp 155–171 | Cite as

Differential Effects of Two Types of Formative Assessment in Predicting Performance of First-year Medical Students

  • Sally KrasneEmail author
  • Paul F. Wimmers
  • Anju Relan
  • Thomas A. Drake
Article

Abstract

Formative assessments are systematically designed instructional interventions to assess and provide feedback on students’ strengths and weaknesses in the course of teaching and learning. Despite their known benefits to student attitudes and learning, medical school curricula have been slow to integrate such assessments into the curriculum. This study investigates how performance on two different modes of formative assessment relate to each other and to performance on summative assessments in an integrated, medical-school environment. Two types of formative assessment were administered to 146 first-year medical students each week over 8 weeks: a timed, closed-book component to assess factual recall and image recognition, and an un-timed, open-book component to assess higher order reasoning including the ability to identify and access appropriate resources and to integrate and apply knowledge. Analogous summative assessments were administered in the ninth week. Models relating formative and summative assessment performance were tested using Structural Equation Modeling. Two latent variables underlying achievement on formative and summative assessments could be identified; a “formative-assessment factor” and a “summative-assessment factor,” with the former predicting the latter. A latent variable underlying achievement on open-book formative assessments was highly predictive of achievement on both open- and closed-book summative assessments, whereas a latent variable underlying closed-book assessments only predicted performance on the closed-book summative assessment. Formative assessments can be used as effective predictive tools of summative performance in medical school. Open-book, un-timed assessments of higher order processes appeared to be better predictors of overall summative performance than closed-book, timed assessments of factual recall and image recognition.

Key words

assessment/testing curriculum development/evaluation educational measurement on-line testing student evaluation teaching methods undergraduate medical education 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. AAMC2004Educating Doctors to Provide High Quality Medical Care: A Vision for Medical Education in the United StatesAAMC Institute for Improving Medical EducationWashington, DCGoogle Scholar
  2. Bentler, P. 2003EQS [Computer Software]. 6.1 BetaMultivariate Software, IncEncino, CAGoogle Scholar
  3. Bondemark, L., Knutsson, K., Brown, G. 2004A self-directed summative examination in problem-based learning in dentistry: A new approachMedical Teacher264651CrossRefGoogle Scholar
  4. Byrne, B.M. 1994Structural Equation Modeling with EQS/Windows: Basic Concepts, Applications, and ProgrammingSage PublicationsThousand Oaks, CalifGoogle Scholar
  5. Ebbinghaus, H. 1885/1913Memory: A Contribution to Experimental PsychologyTeachers College, Columbia UniversityNew YorkGoogle Scholar
  6. Feldhusen, J.F. 1961An evaluation of college students’ reactions to open-book examinationsEducational and Psychological Measurement21637645Google Scholar
  7. Gay, L.R. 1980The comparative effects of multiple-choice versus short-answer tests on retentionJournal of Educational Measurement174550CrossRefGoogle Scholar
  8. Greer, L. 2001Does changing the method of assessment of a module improve the performance of a student?Assessment & Evaluation in Higher Education26127138CrossRefGoogle Scholar
  9. Heck, J.L., Stout, D.E. 1998Multiple-Choice vs. Open-Ended exam problems: Evidence of their impact on student performance in introductory financeFinancial Practice and Education88393Google Scholar
  10. Henly, D. 2003Use of Web-based formative assessment to support student learning in a metabolism/nutrition unitEuropean Journal of Dental Education7116122CrossRefGoogle Scholar
  11. Hill, D.A., Guinea, A.I., McCarthy, W.H. 1994Formative assessment: a student perspectiveMedical Education28394399CrossRefGoogle Scholar
  12. Houghton, G., Wall, D. 2000Trainers’ evaluations of the West Midlands formative assessment package for GP registrar assessmentMedical Teacher22399405CrossRefGoogle Scholar
  13. Hu, L., Bentler, P. 1999Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternativesStructural Equation Modeling6131CrossRefGoogle Scholar
  14. Jehu, D., Picton, C.J., Cher, S. 1970The use of notes in examinationsBritish Journal of Educational Psychology40353357Google Scholar
  15. Marsh, H.W., Hau, K.T., Wen, Z. 2004In search of golden rules: comment on hypothesis-testing approaches to setting cutoff values for fit indexes, dangers in overgeneralizing Hu and Bentler’s (1999) findingsStructural Equation Modeling11320341CrossRefGoogle Scholar
  16. Marton, F., Saljo, R. 1976On qualitative differences in learning: 1. Outcome and processBritish Journal of Educational Psychology46411Google Scholar
  17. Mennin, S.P., Kalishman, S. 1998Student assessmentAcademic Medicine73S46S54Google Scholar
  18. Michaels, S., Kieran, T.R. 1973An investigation of open-book and closed-book examinations in MathematicsThe Alberta Journal of Educational Research19202207Google Scholar
  19. Paschal, C. 2002Formative assessment in Physiology teaching using a wireless classroom communication systemFormative assessment in Physiology teaching using a wireless classroom communication system26299308Google Scholar
  20. Peat, M., Franklin, S. 2002Supporting student learning: The use of computer-based formative assessment modulesBritish Journal of Educational Technology33515523CrossRefGoogle Scholar
  21. Peitzman, S.J., Nieman, L.Z., Gracely, E.J. 1990Comparison of “fact recall” and “higher order” questions in multiple choice examination as predictors of clinical performance of medical studentsAcademic Medicine65S59S60CrossRefGoogle Scholar
  22. Ramsden, P. 1992Learning to Teach in Higher EducationRoutledge, Ch 5LondonGoogle Scholar
  23. Ritter, L. 2000The quest for an effective form of assessment: The evolution and evaluation of a Controlled Assessment Procedure (CAP)Assessment & Evaluation in Higher Education25307320Google Scholar
  24. Rolfe, I., McPherson, J. 1995Formative assessment: how am I doing?The Lancet345837839CrossRefGoogle Scholar
  25. Sadler, D. 1989Formative assessment and the design of instructional systemsInstructional Science18119144CrossRefGoogle Scholar
  26. Schuwirth, L.W.T., Vleuten, C.P.M. 2003ABC of learning and teaching in medicine: Written assessmentBritish Medical Journal326643645Google Scholar
  27. Scouller, K. 1998The influence of assessment method on students’ learning approaches: Multiple choice question examinations versus assignment essayHigher Education35453472CrossRefGoogle Scholar
  28. Smith, S.N. 2005Learning approaches: Examination type, discipline of study, and genderEducational Psychology254353CrossRefGoogle Scholar
  29. Tang, K.C.C. 1992Perception of task demand, strategy attributions and student learningResearch and Development in Higher Education15474481Google Scholar
  30. Thissen-Roe, A., Hunt, E., Minstrell, J. 2004The DIAGNOSER project: Combining assessment and learningBehavior Research Methods, Instruments & Computers36234240Google Scholar
  31. Vaz, M., Avadhany, S.T., Rao, B.S. 1996Student perspectives on the role of formative assessment in physiologyMedical Teacher18324326Google Scholar
  32. Velan, G., Kumar, R., Dziegielewski, M., Walkefield, D. 2003Web-based assessments in pathology with Questionmark PerceptionPathology34282284CrossRefGoogle Scholar
  33. Wass, V., Vleuten, C., Shatzer, J., Jones, R. 2001Assessment of clinical competenceThe Lancet357945949CrossRefGoogle Scholar
  34. Weber, L.J., McBee, K., Krebs, J.E. 1983Take home tests: an experimental studyResearch in Higher Education18473483CrossRefGoogle Scholar

Copyright information

© Springer 2006

Authors and Affiliations

  • Sally Krasne
    • 1
    Email author
  • Paul F. Wimmers
    • 2
  • Anju Relan
    • 3
  • Thomas A. Drake
    • 4
  1. 1.Department of Physiology, David Geffen School of Medicine at UCLAUniversity of CaliforniaLos AngelesUSA
  2. 2.University of Pittsburgh School of MedicinePittsburgh
  3. 3.IDTU, David Geffen School of Medicine at UCLAUniversity of CaliforniaCAUSA
  4. 4.Department of Pathology, David Geffen School of Medicine at UCLAUniversity of CaliforniaLos AngelesUSA

Personalised recommendations