Advances in Health Sciences Education

, Volume 10, Issue 2, pp 145–155

The Role of a Computerised Case-based Testing Procedure in Practice Performance Assessment

Authors

  • L. Schuwirth
    • Department of Educational Development and ResearchMaastricht University
    • Department of Educational Development and ResearchMaastricht University
  • S. Gorter
    • Department of Educational Development and ResearchMaastricht University
  • D. Van der. Heijde
    • Department of RheumatologyAcademic Hospital Maastricht
  • J. J. Rethans
    • Skills laboratoryMaastricht University
  • J. Brauer
    • Department of RheumatologyGeneral Hospital of Heerlen
  • H. Houben
    • Department of RheumatologyGeneral Hospital of Heerlen
  • S. Van der. Linden
    • Department of RheumatologyAcademic Hospital Maastricht
  • C. Van der. Vleuten
    • Department of Educational Development and ResearchMaastricht University
  • A. Scherpbier
    • Educational bureau, Faculty of MedicineMaastricht University
Article

DOI: 10.1007/s10459-004-2784-9

Cite this article as:
Schuwirth, L., Gorter, S., Heijde, D.V.d. et al. Adv Health Sci Educ Theory Pract (2005) 10: 145. doi:10.1007/s10459-004-2784-9

Abstract

Introduction: For postgraduate training of doctors there is a need for valid and reliable instruments to assess their daily performance. Various instruments have been suggested, some of which use incognito simulated patients (SPs). These methods are resource intensive. Computerised Case-based testing (CCT) is logistically simpler and may still predict performance well. The research question was to evaluate the predictive validity of CCT for performance.

Methods: Seventeen rheumatologists were each visited by eight incognito SPs presenting various rheumatological complaints, and scoring the performance of the rheumatologists using a predefined checklist. From this checklist a panel of experts identified essential items. In addition the rheumatologists sat a CCT test containing 55 cases with a total of 121 items.

Results: Negative correlations were found between the SP scores and the CCT scores. This was unexpected. Therefore, background variables on experience were used to compare both methods. The correlation between these and CCT were high and positive and with the SP scores high and negative. This pattern did not differ when using the essential items of the checklist. Reliabilities of the SP scores were markedly high.

Discussion: Although CCT was not predictive of SP scores, it was related to working experience. There are good reasons to assume that although SP-scores were more authentic, they were less valid than CCT scores, mainly because they focussed more on thoroughness than on efficiency in data gathering. The results underpin the assumption that for valid performance assessment the most important issue is what information about the candidate is collected and now how authentic the method is.

Keywords

assessment authentic assessment incognito simulated patients performance assessment rheumatology validity

Copyright information

© Springer 2005