Abstract
A method for evaluating qualitative proficiency testing (PT) of laboratories conducting binary tests is proposed. The method is based on the scale-invariant item response model proposed by the authors in earlier publications. We consider the case where the laboratories under the PT conduct test consisting of a set of test items/species presenting different, but unknown beforehand levels of difficulty when trying to detect a particular property of theirs, and we need to evaluate/compare both the intrinsic abilities of the participating laboratories and the level of difficulty of the test items. We assume that the responses to different test items do not affect one another and discuss how to get and interpret the most likely estimates/scores. The method is illustrated by the example presented in a recent publication by our colleagues from QuoData GmbH and can be considered as an alternative to that proposed in their publication method of scoring.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
AOAC (Association of Official Agricultural Chemists) Official Methods of Analysis (2013) Guidelines for validation of qualitative binary chemistry methods. International Stakeholder Panel on Alternative Methods (ISPAM)
Bashkansky E, Gadrich T (2013) Some statistical aspects of binary measuring systems. Measurement 46:1922–1927
Wehling P, LaBudde RA, Brunelle SL, Nelson MT (2011) Probability of detection (POD) as a statistical model for the validation of qualitative methods. J AOAC Int 94(1):335–347
Wilrich C, Wilrich PT (2009) Estimation of the POD function and the LOD of a qualitative microbiological measurement method. J AOAC Int 92(6):1763–1772
Wilrich P-Th (2010) The determination of precision of qualitative measurement methods by inter-laboratory experiments. Accred Qual Assur 15:439–444
Altman DG, Bland JM (1994) Diagnostic tests. 1: sensitivity and specificity. BMJ 308(6943):1552. doi:10.1136/bmj.308.6943.1552
Uhlig S, Krugener S, Gowik P (2013) A new profile likelihood confidence interval for the mean probability of detection in collaborative studies of binary test methods. Accred Qual Assur 18:367–372. doi:10.1007/s00769-013-0993-8
Pendrill L, Fisher W (2015) Counting and quantification: comparing psychometric and metrological perspectives on visual perceptions of numbers. Measurement. doi:10.1016/j.measurement.2015.04.010
Uhlig S et al (2015) Qualitative PT data analysis with easy-to-interpret scores. Accred Qual Assur 20:347–353. doi:10.1007/s00769-015-1174-8
ISO 13528 (2015) Statistical methods for use in proficiency testing by interlaboratory comparison. International Organization for Standardization, Geneva
Turetsky V, Bashkansky E (2016) Testing and evaluating one-dimensional latent ability. Measurement 78:348–357. doi:10.1016/j.measurement.2015.05.048
Wright BD, Masters GN (1982) Rating scale analysis: Rasch measurement. MESA Press, Chicago
Lee PM (2004) Bayesian statistics: an introduction, 3rd edn. Arnold, London
Acknowledgments
The authors express their deep gratitude to Dr. Ilya Kuselman, who pointed out to us the problem discussed in the article and his continuing interest in the resolution of the issue. We also express our deep appreciation to the anonymous referees, whose highly professional comments contributed to the improvement of the article.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bashkansky, E., Turetsky, V. Proficiency testing: binary data analysis. Accred Qual Assur 21, 265–270 (2016). https://doi.org/10.1007/s00769-016-1208-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00769-016-1208-x