Using uncertainty functions to predict and specify the performance of analytical methods

Abstract

In both European legislation relating to the testing of food and the recommendations of the Codex Alimentarius Commission, there is a movement away from specifying particular analytical methods towards specifying performance criteria to which any methods used must adhere. This ‘criteria approach’ has hitherto been based on the features traditionally used to describe analytical performance. This paper proposes replacing the traditional features, namely accuracy, applicability, detection limit and limit of determination, linearity, precision, recovery, selectivity and sensitivity, with a single specification, the uncertainty function, which tells us how the uncertainty varies with concentration. The uncertainty function can be used in two ways, either as a ‘fitness function’, which describes the uncertainty that is fit for purpose, or as a ‘characteristic function’ that describes the performance of a defined method applied to a defined range of test materials. Analytical chemists reporting the outcome of method validations are encouraged to do so in future in terms of the uncertainty function. When no uncertainty function is available, existing traditional information can be used to define one that is suitable for ‘off-the-shelf’ method selection. Some illustrative examples of the use of these functions in methods selection are appended.

Appendix Examples of the application of the recommended procedure

Example 1. Short concentration range, well above detection limit, no collaborative trial data available

Scenario

The analyte concentration is always in the range 40–50% m/m.

The fitness function

The customer requires a standard uncertainty of 1.5% m/m for fitness for purpose.

The relevant validation information available

The proposed analytical method provides the following, according to validation and information derived from internal quality control (IQC). (All results are % m/m.)

• Repeat analyses of a typical test material (Material 1) within run gives $\bar x=41.6,s_{\rm r}=0.52$

• The detection limit is estimated at 0.02.

• Use of two materials for IQC implied the following statistics:

  Material 2: $\bar x=25.3,\,s_{{\rm run}}=0.41$

  Material 3: $\bar x=52.3,\,s_{{\rm run}}=0.76$

• Analysis of ten spiked test materials estimated that the recovery of the analyte at the appropriate concentration was 95±2% relative.

Building the characteristic function

• As the relevant concentration range is small it is reasonable to regard the uncertainty as invariant.

• Material 1 is within the defined concentration range and by Eq. (2) gives us an estimate of $\hat\sigma_{\rm R}=0.52/0.5=1.04$.

• Material 2 is out of range and therefore should be ignored but, in fact, it provides an RSD similar to that of Material 3 (as expected when the concentration is well above the detection limit), and this adds confidence to the $\hat\sigma_{\rm R}$ value derived from Material 3.

• Material 3 is just over range, so it might give an estimate somewhat on the high side, but in fact (by Eq. (3)) gives $\hat\sigma_{R}=0.76/0.8=0.95$.

• The Horwitz function (Eq. (4)) at a concentration of 50% m/m gives a reproducibility estimate of σ _H =1.1, which is consistent with the above $\hat\sigma_{R}$ estimates and reinforces our confidence in them.

• The detection limit is far below the required range so the zero-point uncertainty makes a negligible contribution to the uncertainty and is ignored.

• We therefore use a consensus of the concordant results for Material 1 and Material 3 to give $\hat\sigma_{R}=1.0$ as the skeleton function.

• We build into the uncertainty an allowance for the uncertainty on the recovery factor. The uncertainty expected on the recovery-corrected mid-range result (45% m/m) is therefore

$$ u_{c}=45\sqrt{\left({\frac{{1.0}}{{45}}}\right)^2 + \left({\frac{2}{{95}}}\right)^2}=1.4. $$

Comparison of uncertainty functions

As u _c <u _f , the method is deemed suitable.

Example 2. Extended concentration range, no collaborative trial data.

Scenario

A commodity is being sold on the basis that the concentration of a particular constituent falls between 5 and 50 mg kg⁻¹. Experience has shown that batches analysed can have levels of the constituent over a somewhat wider range than that.

The fitness function

Preliminary considerations suggest that an RSU of about 10% over the range indicated, and down to 2 mg kg⁻¹, would meet requirements, so the fitness function is $u_f=0.1c,\,2 \le c \le 50$.

The relevant validation information available about the candidate method

• The instrumental detection limit of the method gleaned from the literature was 0.25 mg kg⁻¹ when adjusted for the dilution of the test portion after chemical treatment.

• The repeatability standard deviation, estimated from 10 individual test portions of two putatively typical test materials in a single run, was reported as follows:

  $$ \quad\bar x_{A}=31,\quad s_{A}=1.4,\quad {\rm RSD}=0.045; $$
  $$ \quad\bar x_{B}=103,\quad s_{B}=3.4,\quad{\rm RSD}=0.033. $$

• Single determinations of the analyte in five certified reference materials provided the following results.

  Certified value	Uncertainty on certified value	Reported value
  8.0	0.4	7.2
  108	5.1	101.3
  21.5	0.8	22.7
  42.1	0.9	40.9
  20.2	1.0	20.4

Building the characteristic function

• Realistic detection limits are often much higher than instrumental values, and we adopt the arbitrary decision to raise it by a factor of five to 1.25 mg kg⁻¹. (This is consistent with what little information exists.) As a baseline standard deviation, its contribution is 1.25/2=0.625, and that may not be negligible at the low end of the concentration range of interest, so it will be included in the model.

• The RSDs of the two materials are comparable, and the lower concentration is apparently about 100 times the detection limit, so it is a reasonable assumption that both are estimates of the same constant RSD, so we adopt the mean of the two RSDs (namely 0.039) multiplied by 2.0 as the likely value of A in Eq. (5), (6) etc. (Note: the factor of 2.0 is derived from the application of Eq. (2), because the reported standard deviations were estimated under repeatability conditions.) Under these assumptions, the skeleton characteristic function is given by

  $$ u_c=\sqrt {0.625^2 + 0.078^2 c^2}. $$

• This is reasonably consistent with predictions from the Horwitz function. For example, at 10 mg/kg kg⁻¹, the predicted standard uncertainties are u _c =1.0 and σ _H =1.1, while at 50 mg kg⁻¹ u _c =3.9 and σ _H =4.4 mg kg⁻¹. Accordingly we feel confident in using the skeleton function.

• Considering the results on the reference materials, a reasonable way of summarising them is obtained by looking at the relative differences between the certified values and the found values, i.e., ${{\left({x_{{\rm found}} - x_{{\rm cert}}}\right)}\mathord{/{\vphantom{{\left({x_{{\rm found}} - x_{{\rm cert}}}\right)}{x_{{\rm cert}}}}}\kern-\nulldelimiterspace}{x_{{\rm cert}}}}$ which gives the following (in the tabulated order): −0.10; −0.06; 0.08; −0.03; 0.02. If the mean of these values were significantly different from zero (indicating evidence of overall bias) or the standard deviation were substantially greater than the relative uncertainty expected, the result would suggest that there was an additional source of uncertainty, perhaps due to matrix variation, that needed to be taken into account. In fact the mean relative deviation is zero and the standard deviation is 0.053. The expected relative standard uncertainty (RSU) is obtained by combining the mean RSU of the certified values (0.04) with the relative standard deviation of run-to-run precision, which is estimated as 0.8×0.078=0.062. Thus the expected RSU is$\sqrt {0.053^2 + 0.062^2}=0.08$. This is greater than the observed value, so there are no compelling grounds for inflating the skeleton function. (In fact we might harbour suspicions that the results on the reference materials were unnaturally accurate.)

• This gives us a final characteristic function of

  $$ u_{c}=\sqrt{0.625^2 + 0.078^2 c^2} $$

Comparison of uncertainty functions

Figure 1 shows the plots of u _c and u _f . Over most of the designated concentration range, the characteristic function is below the fitness function, showing that the proposed analytical method could apparently deliver the required degree of accuracy. However, at concentrations below about 10 mg kg⁻¹, the characteristic function is too high, as a consequence of the baseline variability. In order to meet the fitness-for-purpose requirement over the whole range, a method with a lower detection limit would be required.

Fig. 2

Collaborative trial results (circles) from Example 3, with fitted characteristic function and Horwitz function

Fig. 3

Fitness and characteristic functions, Example 3

Example 3. Extended concentration range, collaborative trial data available

Scenario

The requirement addresses the determination of a trace constituent that usually occurs at concentrations between 10 and 100 mg kg⁻¹. From general experience with similar tasks, the nature of the test material and of the proposed method of determination together suggest that uncertainty due to matrix variation within the defined class would be restricted to less than 5% of the concentration. Recovery is expected to be 100%, because there is no scope for loss of analyte.

The fitness function

The client specifies that the uncertainty on the result should not exceed 10% of the concentration or 5 mg kg⁻¹, whichever is the greater.

The relevant validation information available

A collaborative trial has been carried out and the results are as follows.

Test material	Concentration	σ R
A	16.0	1.2
B	31.4	2.0
C	39.8	2.5
D	42.9	3.7
E	46.6	3.8
F	57.1	3.4
G	63.2	4.4
H	69.9	4.0
I	88.6	7.1
J	94.3	5.1

Building the characteristic function

A plot of the collaborative trial data shows the trend of the σ_R as increasing with the concentration of the analyte (Fig. 2). Fitting the data to Eq. (6) provides the estimates σ _R (0)=1.3, c _L =2.6 and A=0.064. The resulting relationship, the skeleton characteristic function, is given by

$$ u_{c}=\sqrt {1.68 + 0.0041c^2} .$$

(The Horwitz function is shown for comparison, and predicts somewhat higher σ_R than observed. In this instance we can ignore the discrepancy and simply use the collaborative trial data.) We now incorporate an extra rotational uncertainty to account for matrix variations equivalent to the maximum thought likely in this analytical system, that is, 5% relative, which, using Eq. (7) gives the final characteristic function

$$\displaylines{ u_{c}=\sqrt{1.68 + ({0.05^2 + 0.0041})c^2}\quad{\rm or}\,\cr u_{c}=\sqrt{1.68 + 0.0066c^2}.}$$

Comparison of uncertainty functions

The characteristic function and the fitness function are shown in Fig. 3. The characteristic function is seen to be the lower over the whole of the relevant range (10–100 mg kg⁻¹), so the method is apparently fit for purpose.