Advances in Health Sciences Education

, Volume 6, Issue 2, pp 103–119

Predicting Holistic Ratings of Written Performance Assessments from Analytic Scoring



The use of experts to judge performance assessments is desirable because ratings of performances, carried out by experts in the content domain of the examination, are often considered to be the ``gold standard.'' However, one drawback of using experts to rate performances is the high cost involved. A more economic alternative for scoring performance assessments entails using analytic scoring, which typically involves assigning points to individual traits present in the performance, and summing to arrive at a single score. This strategy is less costly, but may lack the richness of holistic scoring. This study investigates the use of regression-based techniques to predict expert judgments on a written performance task from a combination of analytic scores. Potentially, this will result in scores that approximate the richness of holistic ratings while maintaining the cost-effectiveness of analytic scoring. Results show that a substantial proportion of variance in expert judgments can be explained by the analytic scores, but that decisions based on actual expert judgments and the predicted expert judgments were not sufficiently consistent to warrant the substitution of one score for the other.

analytic scoring clinical skills assessment holistic rating performance assessment predicting expert judgments 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Kluwer Academic Publishers 2001

Authors and Affiliations

  1. 1.University of Massachusetts AmherstAmherstUSA

Personalised recommendations