Qualitative PT data analysis with easy-to-interpret scores

Abstract

Numerical values called L-scores, with properties similar to those of z-scores, can be calculated for proficiency tests involving qualitative test methods. These scores do not require the use of replicates and are defined in such a way as to reflect both the Level of Competence of the Laboratory (LCL) and the Level of Difficulty of the Task (LDT). On the basis of the “logit” statistical model, the parameters corresponding to the LCL and the LDT values are determined via maximum likelihood estimation. With p denoting the corrected binary response, the model can be interpreted as $\ln \left( {p/\left( {1 - p} \right)} \right) = LCL - LDT.$ The estimated parameters are then the quantities used to compute the L-scores for the participating laboratories. In the case of the laboratory-specific L-scores across tasks, the score for a particular laboratory can be computed as the deviation of the individual LCL from the average LCL relative to the standard error. The interpretation of the L-scores is slightly different from that of z-scores If |L| ≤ 2, the competence of the participant is not significantly different from the average competence. If L > 2, the competence is significantly higher, while if L < −2, the competence is significantly lower.