The Role of a Computerised Case-based Testing Procedure in Practice Performance Assessment
- 106 Downloads
Introduction: For postgraduate training of doctors there is a need for valid and reliable instruments to assess their daily performance. Various instruments have been suggested, some of which use incognito simulated patients (SPs). These methods are resource intensive. Computerised Case-based testing (CCT) is logistically simpler and may still predict performance well. The research question was to evaluate the predictive validity of CCT for performance.
Methods: Seventeen rheumatologists were each visited by eight incognito SPs presenting various rheumatological complaints, and scoring the performance of the rheumatologists using a predefined checklist. From this checklist a panel of experts identified essential items. In addition the rheumatologists sat a CCT test containing 55 cases with a total of 121 items.
Results: Negative correlations were found between the SP scores and the CCT scores. This was unexpected. Therefore, background variables on experience were used to compare both methods. The correlation between these and CCT were high and positive and with the SP scores high and negative. This pattern did not differ when using the essential items of the checklist. Reliabilities of the SP scores were markedly high.
Discussion: Although CCT was not predictive of SP scores, it was related to working experience. There are good reasons to assume that although SP-scores were more authentic, they were less valid than CCT scores, mainly because they focussed more on thoroughness than on efficiency in data gathering. The results underpin the assumption that for valid performance assessment the most important issue is what information about the candidate is collected and now how authentic the method is.
Keywordsassessment authentic assessment incognito simulated patients performance assessment rheumatology validity
Unable to display preview. Download preview PDF.
- Bordage G. (1987). An alternative approach to PMP’s: the “key-features” concept. In: Hart I.R., Harden R. (ed). Further Developments in Assessing Clinical Competence. Proceedings of the Second Ottawa Conference. Can-Heal Publications Inc, Montreal, pp. 59–75Google Scholar
- Chi, M.T.H., Glaser, R., Rees, E. 1982
Expertise in problem solvingSternberg, R.J. eds. Advances in the Psychology of Human Intelligence.Lawrence Erlbaum AssociatesHillsdale, NJ776Google Scholar
- Glaser, R., Chi, M.T.H. 1988
OverviewChi, M.T.H.Glaser, R.Farr, M.J. eds. The Nature of Expertise, xv–xxviii.Lawrence Erlbaum Associates Inc.Hillsdale, NJ, USGoogle Scholar
- Gorter, S., Linden, S., Brauer, J., Vander Heijde, D.H.H., Rethans, J.-J., Scherpbier, A., Vleuten, C., Boonen, A., Dinant, D., Goei The, H., Griep, E., Horst-Bruinsma, I., Linssen, A., Santen-Hoeufft, M., Tempel, H., Westgeest, T. 2001aRheumatologists’ performance in daily practiceArthritis Rheum451627CrossRefGoogle Scholar
- Gorter, S., Rethans, J.J., Scherpbier, A.J.J.A., Linden, S., Van Santen-Hoeufft, M.H.M., Vander Heijde, D.M.F.M., Houben, H.H.M.L., Vander Vleuten, C.P.M. 2001bHow to introduce incognito standardized patients into outpatient clinics of specialists in rheumatologyMedical Teacher23138144CrossRefGoogle Scholar
- Page G., Bordage G., Harasym P., Bowmer I., Swanson D.B. (1990). A new approach to assessing clinical problem-solving skills by written examination: Conceptual basis and initial pilot test results. In: Bender W., Hiemstra R.J., Scherpbier A., Zwierstra R.J. (ed). Teaching and Assessing Clinical Competence. Proceedings of the Fourth Ottawa Conference. Groningen: Boekwerk Publications, Groningen, The Netherlands, (403–407).Google Scholar
- Polsen, P., Jeffries, R. 1982
Expertise in problem solvingSternberg, R.J. eds. Advances in the Psychology of Human Intelligence.Lawrence Erlbaum AssociatesHillsdale, NJ367411Google Scholar
- Posner, M.I. 1988
What is it to be an expert?Chi, M.T.H.Glaser, R.Farr, M.J. eds. The Nature of Expertise, xxix–xxxvi.Lawrence Erlbaum Associates Inc.Hillsdale, NJ, USGoogle Scholar
- Ram, P. 1998Comprehensive assessment of general practitioners. Unpublished dissertationUniversity of MaastrichtMaastricht, The NetherlandsGoogle Scholar
- Schuwirth, L.W.T. 1998An approach to the assessment of medical problem solving: Computerised Case-based Testing (dissertation)University of MaastrichtMaastrichtGoogle Scholar
- Schuwirth, L.W.T., Vander Vleuten, C.P.M., De Kock, CA, Peperkamp, A.G.W., Donkers, H.H.L.M. 1996Computerized case-based testing: a modern method to assess clinical decision makingMedical Teacher.18295300Google Scholar
- Swanson, D.B. 1987
A measurement framework for performance-based testsHart, I.Harden, R. eds. Further developments in Assessing Clinical Competence.Can-Heal PublicationsMontreal1345Google Scholar
- Swanson, D.B., Norcini, J.J., Grosso, L.J. 1987Assessment of clinical competence: written and computer-based simulationsAssessment and Evaluation in Higher Education.12220246Google Scholar