Response from the Analytical Methods Committee (AMC) to the paper “Uncertainty in repeated measurement of a small non-negative quantity: explanation and discussion of Bayesian methodology”

Abstract

Willink’s paper, “Uncertainty in repeated measurement of a small non-negative quantity: explanation and discussion of Bayesian methodology” provides a clear exposition of the Bayesian paradigm but unfortunately reads like a rebuttal of a previous paper from the AMC: the reader is left with the impression that the AMC is recommending a procedure that performs poorly. The AMC here refutes any such impression by presenting a balanced assessment of the strengths and weaknesses of the Bayesian and frequentist approaches to formulating the uncertainty of small non-negative quantities.

Appendix: Current AMC recommendation

The following procedure is recommended for providing results near natural limits:

• 1. Calculate the combined standard uncertainty and degrees of freedom, and apply the relevant coverage factor without regard to natural limits at any stage of the calculation.

• 2. Determine (usually from information gathered in contracting for the work) whether the customer requires a summary of raw observations for subsequent analysis, or an inference about the true analyte concentration for the particular test material.

• 3. If the customer requires raw observations, report the observed mean and classical confidence interval without regard to the natural limits, even if part or all of the interval is outside the possible range.

• 4. If the customer requires an estimate of the true analyte concentration, truncate any part of the expanded interval which lies outside the possible range. If the observed mean is out of range, move it to the nearest limit.

• 5. If the original expanded uncertainty is so out of range that it is completely beyond the natural limit, investigate the original experiment and repeat if necessary.

• 6. Retain (and report if required) the original standard uncertainty and degrees of freedom.

An example of a complete report format following these rules and containing all the relevant information would be: “estimated value 0.995 with 95% confidence interval 0.983–1.000 based on a standard uncertainty of 0.005 and 11 degrees of freedom”.

Finally, never truncate or censor result which will be used in subsequent calculations or statistical analysis. Only truncate or adjust results to provide the final inference drawn from the observations.