The Swiss Way to Score Multiple True-False Items: Theoretical and Empirical Evidence

Summary

In medical problems — dependent on the available information — there is often more than one reasonable solution. A disorder can have different causes, in the beginning of a differential diagnosis several possibilities must be taken into consideration, often it is reasonable to take more than one diagnostic and/or therapeutic measure. To test the performance in such situations, multiple true-false items are more convenient than MCQs with only one correct answer.

Commonly, in true-false items the candidates are either credited for each single correct decision (Type X) or they have to choose one out of five presented response patterns (Type K). From the psychometric point of view, both versions are problematic, among other things due to the guessing influence. In the Swiss medical examinations true-false items with four decisions are used since the early seventies. A I point credit is only given for a fully correct response pattern (All-or-nothing evaluation). Since 1988 we also use a variation with a half-point credit for three correct decisions.

Theoretical and empirical evidence for the use of this item type and the two scoring versions are presented.