Efficient estimation of the limit of detection and the relative limit of detection along with their reproducibility in the validation of qualitative microbiological methods by means of generalized linear mixed models
- 487 Downloads
Abstract
The reproducibility of measurement results is a core performance characteristic for quantitative methods. However, in the validation of qualitative methods it is not clear how to characterize a method’s reproducibility. One approach for determining a qualitative method’s reproducibility is presented for microbiological methods, where the distribution of colony forming units (CFU) follows a Poisson distribution. The method’s reproducibility is defined in terms of the variability of the limit of detection (LOD) values. For a better estimation of reproducibility precision, our proposed approach is using an orthogonal factorial plan. Since an exact determination of absolute contamination levels is often not possible, following the ISO 16140-2:2016 [Microbiology of food and animal feed—method validation—part 2: protocol for the validation of alternative (proprietary) methods against a reference method, 2016], an approach is proposed which is based on the ratio of the LOD values of a reference and an alternative method. This approach is illustrated on the basis of an example.
Keywords
Limit of detection Relative limit of detection Validation Microbiology Methods Factorial study1 Introduction
An appropriate approach for the validation of qualitative methods will often differ considerably from that of quantitative methods. Nevertheless, core concepts from the validation of quantitative methods can be successfully carried over to qualitative methods. This paper shows how the reproducibility of a method—a performance characteristic usually associated with quantitative methods—can be determined in collaborative studies for qualitative methods in microbiology.
In analytical chemistry, one of the fundamental indicators of the performance of a quantitative method is the reproducibility of test results, as described in ISO 5725 (ISO 1994). While the concept of reproducibility is easily interpreted for qualitative methods in terms of consistent test results across laboratories for samples with the same level of contamination, it is not clear at all how to describe or characterize a qualitative method’s reproducibility in such a way as to make possible a comparison to criteria or other methods. In the last few years, however, novel validation approaches have been proposed for the characterization of the reproducibility of a qualitative method (Uhlig et al. 2011, 2013, 2015; Grohmann et al. 2015).
Why is it important to determine a method’s reproducibility? In order to answer this question, consider the case that a level of detection (LOD) of 3 colony forming units (CFU) per mL is determined in the validation study of a qualitative microbiological method, but that the LOD is sometimes much higher depending on the laboratory or measurement conditions. In such a case, failing to detect the occasional unreliability of the method could lead to mistakes in routine laboratory determinations. On the other hand, if a LOD of 300 CFU/mL is obtained in the validation study, the method will not be accepted even if this excessive LOD is not representative of its average performance. Accordingly, both the average LOD value and the reproducibility parameter—describing the variability of the LOD across laboratories or measurement conditions—capture important information about the performance of the method and should be determined in the course of the validation process.
In the case of microbiological methods, an exact determination of absolute contamination levels is often not possible. For this reason, the ISO 16140-2 (ISO 2016) proposes an approach which is based on the ratio of the LOD values of a reference and an alternative method. Just as in the case of the LOD, both average and reproducibility precision parameters can be calculated for this relative LOD (RLOD) value.
In order to determine the reproducibility of a qualitative method, a suitable approach must be identified for the conversion of the qualitative results into quantitative ones. In this paper, the case will be considered where the distribution of CFU contamination levels follows a Poisson distribution. The reliability and robustness of the validation can be enhanced by means of a systematic study of the effect of influence factors. Such an approach also allows a reduction in workload, with reliable validation parameters with as few as 5 participating laboratories.
2 Materials and methods
The approach presented here is based on the computation of a power curve, which plots the probability of detection POD (probability that the target microorganism is detected) as a function of the contamination level x (in CFU/mL). The limit of detection LOD 95% or LOD 50% is then defined as the contamination level corresponding to POD(LOD 95%) = 0.95 or POD(LOD 50%) = 0.5.
As can be seen, the POD increases with a. The value \( a = 0 \) corresponds to \( POD = 0 \) no matter the nominal number of copies (i.e. the method is useless), while, at the other extreme, the value a = 1 corresponds to \( POD = 1 - \exp ( - x) \) (i.e. the method is perfect).
In practice, it may occur that a i values greater than 1 are observed (or, equivalently, \( \ln a_{i} \ge 0 \)). Theoretically, such an occurrence is not compatible with the Poisson distribution assumption, since, for a given nominal concentration x, the corresponding POD would be greater than \( 1 - \exp \left( { - x} \right) \). Accordingly, it may seem desirable to constrain the sensitivity parameter estimates to values \( a_{i} \le 1 \). However, \( a_{i} > 1 \) can be interpreted as an indication that the average target microorganism concentration is greater than the nominal concentration or that the number of false positives is too large. In the framework of a validation study, this constitutes useful information and, for this reason, it was decided not to build in an extra constraint (note that \( a_{i} > 0 \) is ensured by applying the exponential function to the \( \ln a_{i} \) estimate).
Study design in the case of five factors for each participating laboratory
Summary of ROD values for each participating laboratory
| Contamination level | Setting/run | |||||||
|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | |
| L 0 | ||||||||
| L 1 | ||||||||
| L 2 | ||||||||
Positive results for one method
| Laboratory | Contamination level | ||
|---|---|---|---|
| L0 | L1 | L2 | |
| Laboratory 1 | k 10/n 0 | k 11/n 1 | k 12/n 2 |
| Laboratory 2 | k 20/n 0 | k 21/n 1 | k 22/n 2 |
| Laboratory 3 | k 30/n 0 | k 31/n 1 | k 32/n 2 |
| Etcetera | … | … | … |
| All | … | … | … |
Note that in a validation study, the design matrix elements are constants, i.e. they are not subject to random variation. They are systematically selected in order to reflect the spectrum of measurement conditions in the laboratory. However, in routine measurements no such deliberate control is exercised over measurement conditions, and the z jkl values can be seen as independent realizations of a random variable with zero mean and unit variance.
The within-laboratory effects \( \gamma_{ikl} \) values are modelled as independent normal random effects with \( \gamma_{ikl} \sim {\text{N(}}0,\sigma_{k}^{2} ) \).
The \( \sigma_{total}^{2} \) parameter thus characterizes the reproducibility of the method.
This establishes a direct relationship between the average sensitivity a [calculated as e μ , see Eq. (5)] and LOD 95%. Thus, in the ideal case (a = 1), we obtain \( LOD_{95\% } \cong 3. \) On the other hand, if the sensitivity parameter a drops to 1/2, LOD 95% increases to \( \cong 6 \).
One obtains the same result with LOD 50% instead of LOD 95%.
Thus, the (log) reproducibility variability of the LOD 95% (or LOD 50%), defined as the logarithmic ratio between upper and lower 95 % confidence limits, is proportional to σtotal.
Simulation studies were conducted in order to assess the reliability of the σ total estimate. With 5 participant laboratories, a relative standard error of less than 30 % was observed for the σ total estimate. It can thus be concluded that reliable reproducibility estimates are achieved with as few as 5 laboratories.
There are 2 approaches for the determination of the RLOD50%. If the contamination levels are not known, only a direct estimation of RLOD is possible, see Section 5.1.4.2 of ISO 16140-2 (ISO 2016).
3 Results and discussion
Design with five factors and eight settings to be implemented within each laboratory and for each contamination level
Data for example
| Contamination level (CFU/mL) | Setting | Laboratory 1 | Laboratory 2 | Laboratory 3 | Laboratory 4 | Laboratory 5 | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| R1 | R2 | R3 | R4 | R1 | R2 | R3 | R4 | R1 | R2 | R3 | R4 | R1 | R2 | R3 | R4 | R1 | R2 | R3 | R4 | ||
| Blank | 1 | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – |
| 2 | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | |
| 3 | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | |
| 4 | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | |
| 5 | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | |
| 6 | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | |
| 7 | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | |
| 8 | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | |
| 0.8 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 2 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 1 | 0 | 1 | 1 | 0 | 1 | 1 | |
| 3 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | |
| 4 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | |
| 5 | 0 | 1 | 0 | 0 | 1 | 1 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | |
| 6 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 1 | 1 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | |
| 7 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | |
| 8 | 0 | 0 | 1 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | |
| 10 | 1 | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – |
| 2 | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 0 | – | – | – | |
| 3 | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | |
| 4 | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | |
| 5 | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | |
| 6 | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | |
| 7 | 1 | – | – | – | 1 | – | – | – | 0 | – | – | – | 1 | – | – | – | 1 | – | – | – | |
| 8 | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | |
ROD values for laboratory 1
| Setting/run | ||||||||
|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | |
| Factor | ||||||||
| Technician | 1 | 2 | ||||||
| Culture medium | 1 | 2 | 1 | 2 | 1 | 2 | 1 | 2 |
| Thawing process | 1 | 2 | 1 | 2 | 2 | 1 | 2 | 1 |
| Incubator | 1 | 2 | 2 | 1 | 1 | 2 | 2 | 1 |
| Background flora | 1 | 2 | 2 | 1 | 2 | 1 | 1 | 2 |
| Contamination level (CFU/mL) | ||||||||
| Blank | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 0.8 | 0.75 | 0.75 | 0 | 1 | 0.25 | 0.75 | 1 | 0.25 |
| 10 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
The estimation of the model parameters is carried out in the statistical software R. Alternatively, the computations can be performed by means of an extended version of the software PROLab POD (QuoData). The mean sensitivity estimate is 0.61. It follows that LOD 50% is approximately 1.13 [see Eq. (10)]. Finally, σ total is estimated as 0.76. It follows that \( \ln \frac{{LOD_{50\% ,upper} }}{{LOD_{50\% ,lower} }} = 3.92 \times 0.76 = 2.97 \) [see Eq. (12)].
Data for example
| Contamination level (CFU/mL) | Setting | Laboratory 1 | Laboratory 2 | Laboratory 3 | Laboratory 4 | Laboratory 5 | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| R1 | R2 | R3 | R4 | R1 | R2 | R3 | R4 | R1 | R2 | R3 | R4 | R1 | R2 | R3 | R4 | R1 | R2 | R3 | R4 | ||
| Blank | 1 | 0 | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – |
| 2 | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | |
| 3 | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | |
| 4 | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | |
| 5 | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | |
| 6 | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | |
| 7 | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | |
| 8 | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – | |
| 0.8 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 0 |
| 2 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | |
| 3 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | 1 | |
| 4 | 1 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | |
| 5 | 0 | 0 | 1 | 1 | 0 | 0 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 1 | 1 | |
| 6 | 0 | 1 | 0 | 1 | 0 | 1 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 1 | 0 | 1 | 1 | |
| 7 | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 1 | 0 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | |
| 8 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | |
| 10 | 1 | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – |
| 2 | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | |
| 3 | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | |
| 4 | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | |
| 5 | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | |
| 6 | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | |
| 7 | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | |
| 8 | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | 1 | – | – | – | |
Positive results for the alternative method
| Laboratory | Contamination level | ||
|---|---|---|---|
| L 0 = Blank | \( L_{1} = 0.8\,\,{\text{CFU}}/{\text{mL}} \) | \( L_{2} = 10\,\,{\text{CFU}}/{\text{mL}} \) | |
| Laboratory 1 | 0.000 | 0.594 | 1.000 |
| Laboratory 2 | 0.000 | 0.688 | 1.000 |
| Laboratory 3 | 0.000 | 0.250 | 0.875 |
| Laboratory 4 | 0.000 | 0.281 | 1.000 |
| Laboratory 5 | 0.000 | 0.344 | 0.875 |
| All | 0.000 | 0.431 | 0.950 |
Positive results for the reference method
| Laboratory | Contamination level | ||
|---|---|---|---|
| L 0 = Blank | \( L_{1} = 0.8\,\,{\text{CFU}}/{\text{mL}} \) | \( L_{2} = 10\,\,{\text{CFU}}/{\text{mL}} \) | |
| Laboratory 1 | 0.000 | 0.406 | 1.000 |
| Laboratory 2 | 0.000 | 0.344 | 1.000 |
| Laboratory 3 | 0.000 | 0.625 | 1.000 |
| Laboratory 4 | 0.000 | 0.406 | 1.000 |
| Laboratory 5 | 0.000 | 0.563 | 1.000 |
| All | 0.000 | 0.469 | 1.000 |
The LOD of the reference method is calculated as LOD 50%,ref = 0.88. As can be seen, it is lower than that of the alternative method (LOD 50%,alt = 1.13), i.e. the reference method is more sensitive. The corresponding RLOD 50% value is calculated as 1.28 [see Eq. (12)].
\( { \log }_{10} {\text{RLOD}}_{50\% } \) values for each laboratory and setting
| Laboratory | Setting | \( LOD_{50\% ,alt} \) | LOD 50%,ref | \( { \log }_{10} RLOD_{50\% } \) |
|---|---|---|---|---|
| 1 | 1 | 0.69 | 4.60 | −0.82 |
| 2 | 0.69 | 0.80 | −0.06 | |
| 3 | 4.60 | 0.80 | 0.76 | |
| 4 | 0.69 | 1.77 | −0.41 | |
| 5 | 1.77 | 0.80 | 0.35 | |
| 6 | 0.69 | 0.80 | −0.06 | |
| 7 | 0.69 | 1.77 | −0.41 | |
| 8 | 1.77 | 0.69 | 0.41 | |
| 2 | 1 | 0.80 | 0.80 | 0.00 |
| 2 | 0.80 | 0.80 | 0.00 | |
| 3 | 0.69 | 4.60 | −0.82 | |
| 4 | 0.80 | 0.69 | 0.06 | |
| 5 | 0.69 | 1.77 | −0.41 | |
| 6 | 0.69 | 0.80 | −0.06 | |
| 7 | 0.69 | 1.77 | −0.41 | |
| 8 | 0.69 | 4.60 | −0.82 | |
| 3 | 1 | 1.77 | 1.77 | 0.00 |
| 2 | 0.80 | 0.80 | 0.00 | |
| 3 | 1.77 | 0.69 | 0.41 | |
| 4 | 1.77 | 0.69 | 0.41 | |
| 5 | 1.77 | 0.69 | 0.41 | |
| 6 | 0.80 | 0.80 | 0.00 | |
| 7 | 4.60 | 0.80 | 0.76 | |
| 8 | 4.60 | 0.69 | 0.82 | |
| 4 | 1 | 1.77 | 0.80 | 0.35 |
| 2 | 0.80 | 4.60 | −0.76 | |
| 3 | 4.60 | 1.77 | 0.41 | |
| 4 | 0.80 | 0.80 | 0.00 | |
| 5 | 1.77 | 0.80 | 0.35 | |
| 6 | 0.80 | 0.80 | 0.00 | |
| 7 | 1.77 | 0.69 | 0.41 | |
| 8 | 4.60 | 4.60 | 0.00 | |
| 5 | 1 | 4.60 | 1.77 | 0.41 |
| 2 | 2.75 | 0.80 | 0.54 | |
| 3 | 1.77 | 0.69 | 0.41 | |
| 4 | 0.69 | 1.77 | −0.41 | |
| 5 | 4.60 | 0.80 | 0.76 | |
| 6 | 4.60 | 0.69 | 0.82 | |
| 7 | 4.60 | 0.69 | 0.82 | |
| 8 | 0.69 | 0.69 | 0.00 |
4 Conclusions
In this paper, a validation approach is presented for microbiological qualitative methods where the distribution of CFU contamination levels follows a Poisson distribution. In this approach, the method’s reproducibility is a measure of the reproducibility of the LOD parameter across laboratories and measurement conditions. Since a microbiological qualitative method cannot be reliably validated without determining the variability of the LOD, the method’s reproducibility—calculated as σ total —provides essential information about the method’s performance.
Moreover, the factorial design presented here constitutes a systematic approach to measurement conditions which, over and above ensuring the full range of measurement conditions is represented in the validation study, makes it possible to reduce the workload, with reliable reproducibility estimates with as few as 5 laboratories. In addition, the factorial approach also allows a quantitative analysis of the impact of different influence factors.
If, as is often the case for microbiological methods, sufficient stability of the samples is not ensured, then test results from a reference method should be taken into consideration, and the assessment of the reproducibility is carried out with respect to the two methods’ relative level of detection. Since it can be expected that sample instability will affect both methods in the same manner, considering the ratio of the 2 LOD values should offset any bias in the estimate of reproducibility caused by sample instability. The reproducibility of the RLOD parameter only provides information regarding the reproducibility of the LOD of the alternative and reference methods if the two measurement procedures can be considered independent, e.g. involving different culture media, reagents and instruments.
Finally, it needs to be noted that the approach presented here can be adapted to in-house validation studies. The factor “Laboratory” can be replaced by the factor “Day” or “Week”. The variability between the laboratories would then correspond to the variability between days or weeks.
Footnotes
- 1.
The model described here does not include the slope parameter, see ISO 16140-2 (ISO 2016). Indeed, it has been observed that, in the case of culture methods, the slope parameter can usually be omitted.
- 2.
The design matrix codifies which factor levels are associated with a particular test result. Thus, if there are 2 levels per factor, the design matrix contains zero and one (“0” for the one level and “1” for the other level). Note that one could also use a different coding strategy, such as coding the one factor level with “−1” and the other factor level with “1”. The same results would be obtained, but some of the calculations would require slight adjustments [e.g. Eq. (8)].
Notes
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
References
- Grohmann L, Reiting R, Mäde D, Uhlig S, Simon K, Frost K, Randhawa GJ, Zur K (2015) Collaborative trial validation of cry1Ab/Ac and Pubi-cry TaqMan-based real-time PCR assays for detection of DNA derived from genetically modified Bt plant products. Accredit Qual Assur 20:85–96CrossRefGoogle Scholar
- ISO 16140-2:2016 (2016) Microbiology of food and animal feed—method validation—part 2: protocol for the validation of alternative (proprietary) methods against a reference methodGoogle Scholar
- ISO 5725 (1994) Accuracy (trueness and precision) of measurement methods and results—parts 1–6Google Scholar
- Jiang J (2007) Linear and generalized linear mixed models and their applications. Springer, New YorkGoogle Scholar
- McCulloch CE, Searle SR (2001) Generalized, linear, and mixed models. Wiley, New YorkGoogle Scholar
- Nelder JA, McCullagh P (1983) Generalized linear models. Chapman and Hall, CambridgeGoogle Scholar
- Tamhane AC (2009) statistical analysis of designed experiments: theory and applications. Wiley, HobokenCrossRefGoogle Scholar
- Uhlig S, Niewöhner L, Gowik P (2011) Can the usual validation standard series for quantitative methods, ISO 5725, be also applied for qualitative methods? Accredit Qual Assur 16:533–537CrossRefGoogle Scholar
- Uhlig S, Krügener S, Gowik P (2013) A new profile likelihood confidence interval for the mean probability of detection in collaborative studies of binary test methods. Accredit Qual Assur 18:367–372CrossRefGoogle Scholar
- Uhlig S, Frost K, Colson B, Simon K, Mäde D, Reiting R, Gowik P, Grohmann L (2015) Validation of qualitative PCR methods on the basis of mathematical-statistical modelling of the probability of detection. Accredit Qual Assur 20:75–83CrossRefGoogle Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.