Determination of the standard deviation for proficiency assessment from past participant’s performances

Abstract

The “uncertainty function” introduced by Thompson et al. estimates the reproducibility standard deviation as a function of concentration or mass fraction. This model was successfully applied to data derived from three proficiency testing schemes aiming at the quantification of cadmium, lead and mercury in blood and urine. This model allows the estimation of standard deviation for the performance assessment for proficiency testing rounds.

Introduction

The “Centre de toxicologie du Québec” (CTQ) [1] belonging to the “Institut national de santé publique du Québec” (INSPQ) is a public organization that has been offering human toxicology expertise (environmental, clinical and occupational) to the provincial health network of Quebec (Canada) as well as to external clients from around the world. Since 1979, the CTQ operates several permanent external quality assessment schemes that enable participating laboratories to evaluate the accuracy and precision of their analytical methods on a continuous basis. Approximately 250 laboratories from over 30 countries participate in these proficiency testing (PT) schemes to analyse a wide variety of elements in biological PT materials of human origin, such as blood, serum, urine or hair.

In order to verify and further confirm the applicability of the “uncertainty function” described and discussed in several publications [2–8], we compiled all the reference values (X _Ref) and the corresponding reproducibility standard deviations (s _R) determined in the frame of three PT programs designed for the determination among others of three toxic trace elements (cadmium, mercury and lead) in blood and urine matrices. A total of 861 data pairs (X _Ref, s _R)—later denoted as the CTQ data—were analysed to identify whether similar trends are observed in the seventeen cases under investigation (3 PT schemes; 3 elements; 2 matrices; one PT scheme does not monitor mercury in urine).

Methodology

The experimental data evaluated in this work were reported in the frame of the three PT programs described hereafter:

• The “Interlaboratory Comparison Program for metals in biological matrices” (PCI) is a bimonthly scheme attended by over 130 laboratories applying their routine analytical techniques;

• The “Priority Metals Quality Assessment Scheme” (PMQAS) is a scheme designed for the US State Laboratories, all equipped with the same experimental instrumentation (inductive coupled plasma mass spectrometry, ICP-MS) and applying the same experimental protocols for the analysis of trace elements in blood and urine; and

• The “Quebec Multi-element External Quality Assessment Scheme” (QMEQAS) attended by 60 laboratories using ICP-MS.

At the end of the each PT round, a classical statistical treatment was applied to the results reported by the participants to calculate—after outlier rejection—the median value and the reproducibility standard deviation (s _R). The median value was set as the assigned reference value (X _Ref), while s _R was used to derive the standard deviation for performance assessment (σ _PT). The CTQ data were compiled from the previous PT rounds organized by the CTQ for cadmium, mercury and lead in blood and urine matrices, as indicated in Table 1. All values were systematically converted to mass fraction (g g⁻¹).

Table 1 Summary of the CTQ data, including the number of data points (N), the mass fraction ranges investigated (C), the fitted parameters α, β and calculated ratios α/β, for cadmium, mercury and lead in urine and blood, assigned in the frame of the PCI, QMEQAS and PMQAS proficiency testing schemes. The relative standard errors of the parameters and their ratio are indicated between parentheses

In the early 1980s, Horwitz et al. [9] reviewed the reported results in the frame of the Association of Official Analytical Chemists (AOAC) PT rounds and derived an empirical relation estimating the coefficient of variation for the reproducibility (CV _R) as a function of the mass fraction (C) expressed in g g⁻¹:

$$ CV_{\text{R}} = {{2^{(1 - 0.5\lg C)} } \mathord{\left/ {\vphantom {{2^{(1 - 0.5\lg C)} } {100}}} \right. \kern-\nulldelimiterspace} {100}} $$

(1)

Twenty years later, Thompson re-evaluated [10] the results reported in several PT schemes and confirmed the validity of the Horwitz equation at mass fractions ranging from 1.2×10⁻⁷ to 0.138 g g⁻¹, while suggesting a constant CV _R of 0.22 (or 22 %) for mass fractions below 1.2×10⁻⁷ g g⁻¹.

In order to have a clearer view on how to proceed, we plotted a set of four graphs for each combination of PT scheme, element and matrix, namely:

• a. CV _R versus C, as suggested by Horwitz [9];

• b. s _R versus C;

• c. lg(CV _R) versus lg(C), where “lg” denotes the logarithm to base 10; and

• d. lg(s _R) versus lg(C), as suggested by Thompson [7, 10].

An example of such a set of graphs is shown in Fig. 1a–d presenting all the data collected in the frame of the PMQAS round for the determination of cadmium in blood. A constant CV _R of approximately 0.05 is observed at higher concentration (Fig. 1a)—equivalent to a linear increase in s _R with increasing C (Fig. 1b)—while a constant s _R is observed at the lowest mass fraction range (Fig. 1d). These observations are consistent with the “uncertainty function” introduced by Thompson in 1988 [2] and further discussed since [3–7]:

$$ s_{\text{R}} = \sqrt {\alpha^{2} + \beta^{2} C^{2} } $$

(2)

Fig. 1

The four graphical representations of the CTQ data for Cd in blood obtained in the frame of the PMQAS proficiency testing scheme: a CV _R versus C; b s _R versus C; c lg(CV _R) versus lg(C); and d lg(s _R) versus lg(C). The solid line represents the “uncertainty function” fitting the experimental data points. C and s _R are expressed in g g⁻¹

The function described in Eq. 2 was systematically fitted to the CTQ data. The Newton-Raphson algorithm implemented in the Microsoft Excel 2010 Solver was used—without any further data weighting—to minimize the sum of squares of residuals and to derive the two parameters α and β. The initial value of parameter β was set equal to the CV _R of the highest mass fraction investigated, while the initial value of parameter α was set to 10⁻⁹ by default.

Results and discussion

Neither the Horwitz model nor the Thompson-modified one fits the CTQ experimental data obtained for cadmium, mercury or lead. The example for cadmium illustrated in Fig. 2 clearly shows that most of the data lie below the two model curves.

Fig. 2

The CTQ data for cadmium in urine (filled symbols) and blood (empty symbols) assigned in the frame of the PCI (squares), QMEQAS (circles) and PMQAS (triangles) proficiency testing schemes. Most of the data lie below the Horwitz (dashed line) and the Thompson-modified (solid line) model curves. C and s _R are expressed in g g⁻¹

One the other hand, the “uncertainty function” (Eq. 2) fits well the CTQ data. An example is shown in Fig. 1. As stated by Thompson [8], the “uncertainty function” is function of parameter α “[…] describing the constant variation at concentrations close to the detection limit […]” and of parameter β representing “[…] the constant relative standard deviation at high concentration […].” On the basis of this assumption, an alternative mathematical approach was derived from Eq. 2, confirmed the values obtained for α and β, and allowed the estimation of the respective relative standard errors from the corresponding variations, which could not be obtained using the MS Excel 2010 Solver. For each PT-matrix-element combination, an estimate of β was calculated as the average of CV _R at the high concentration range. The mean value of α was then derived as:

$$ \alpha = \overline{{\sqrt {s_{R}^{2} - \beta^{2} C^{2} } }} $$

(3)

When plotting the “uncertainty function” versus mass fraction, one gets the characteristic shape predicted by Horwitz [9]—sometimes referred as the “Horwitz trumpet” [11]—and described by Thompson [2]. Figure 1c and d shows that the “uncertainty function” has two asymptotes—represented by a constant reproducibility standard deviation, below a certain mass fraction; while above it, represented by a constant coefficient of variation of the reproducibility. The two asymptotes intercept at a mass fraction equal to the ratio α/β. Table 1 presents the mass fraction ranges, the values for α, β and the ratio α/β, together with the respective relative standard errors—provided between parentheses—for the seventeen PT-matrix-element combinations investigated.

Reliable β values are determined with a relative standard error ranging from 14 to 23 %. The β values for the PMQAS program are systematically the smallest of the order of 0.05, as expected from a PT scheme having participants using the same experimental protocol and the same instrumentation. The other PT schemes display β values of 0.07, 0.08 and 0.11 for Pb, Cd and Hg, respectively (Table 1). Koch and Magnusson reported similar results [12].

Assuming that the “uncertainty function” remains applicable down to mass fraction close to the limit of quantification (C _LOQ) and to the limit of detection (C _LOD) one could estimate following indicative upper limits:

$$ C_{\text{LOD}} = 3\alpha \;\;{\text{or}}\;\;C_{\text{LOQ}} = 10\alpha $$

(4)

Fewer and more scattered data were available for the determination of α, for which the relative standard errors ranged from 23 to 45 %. α values of 0.2, 1 and 3 μg kg⁻¹ were obtained for Cd, Hg and Pb, respectively (Table 1). This would correspond to estimated limits of detection of 0.6, 3 and 9 μg kg⁻¹ in blood and urine matrices. These limits are well above—up to 20 times—those determined experimentally for a specific sample treatment and a dedicated instrumental technique. Such over-estimated values may be due to the fact that the presented α values derive from reproducibility standard deviations (computed from results reported in the frame of several PT schemes, and obtained using various analytical methods), whereas C _LOD are usually determined under repeatability conditions.

The ratio α/β for each element from the different matrices and PT schemes are in agreement within 20 %, when excluding the value for the PMQAS Cd in urine. Ratios of the order of 2, 8 and 48 μg kg⁻¹ were obtained for Cd, Hg and Pb, respectively (Table 1). When combining with Eq. 4, the following approximations are derived: α/β ≈ 2C _LOQ for β = 0.05 (i.e. PMQAS) or α/β ≈ C _LOQ for β = 0.10 (i.e. PCI or QMEQAS). This indicates that CTQ might have organized some PT rounds close to the limit of quantification, below which measurement relative uncertainties higher than 22 % are to be expected. This could explain the high scatter of data points at the low concentration range.

Conclusion

The “uncertainty function” introduced by Thompson et al. [2] describes well the trend of reproducibility standard deviation versus mass fraction for cadmium, mercury and lead in blood and urine samples. The compilation of α and β calculated using a simple mathematical approach (without any data weighting) will allow CTQ to estimate reproducibility standards deviations and to derive the standard deviation of performance assessment (σ _PT) for various PT-element-matrix combination. The robust statistical treatment prescribed by the ISO 13528 guide [13] would be performed for confirmation. On the other hand, participants could use the same function to calculate the reproducibility standard deviation to derive a reasonable estimate of their measurement uncertainty, as prescribed by the Eurolab [14]. The CTQ intends to evaluate the “uncertainty function” for the remaining elements and matrices available.

Furthermore, the CTQ will evaluate the dispatch of identical PT samples in several PT schemes, using, for example, the assigned values (X _Ref, σ _PT) of one PT round to the other PTs, similar to what is implemented by the International Measurement Evaluation Program (IMEP) [15]. This could significantly reduce the costs for homogeneity and stability investigation, ensuring the propagation of sound metrological principle to various groups of participants.