# Bayesian Estimation for Diagnostic Testing of Biosecurity Risk Material in the Absence of a Gold Standard when Test Data are Incomplete

## Abstract

Diagnostic testing is used by biosecurity officers for the detection and identification of plant and animal pathogens, often informing high-consequence decisions such as restricting the entry of trade goods. It is rare that such tests can be considered gold standards; however, uncertainty can be reduced by using the results of other tests, measuring performance on samples of known status and incorporating prior knowledge from expert judgement. This article presents an extension to the methods of Joseph et al. (Am J Epidemiol 141:263–272, 1995), and Dendukuri and Joseph (Biometrics 57:158–167, 2001) for Bayesian estimation in the absence of a gold standard test, which allows for the use of incomplete test data. This extension is demonstrated with a novel application: the case study of myrtle rust from Holliday et al. (Plant Dis 97:828–834, 2013), which involves samples from potential biosecurity risk material on importation pathways to Australia. The samples were tested at two laboratories, and prior estimates for pathway prevalence were obtained by expert elicitation. The Bayesian estimation was based on a model with and without covariances for the test results to assess the assumption of conditional independence. The results show that pathogen prevalence, diagnostic sensitivity and diagnostic specificity can be estimated using all available data even where some samples have been subject to only one of two available tests. The results also indicate the importance of consideration of the assumption of conditional independence. The findings enable diagnostic testing laboratories and decision makers to make use of all test results and to explicitly incorporate prior knowledge to estimate pathogen prevalence and test accuracy.

### Keywords

Biosecurity Disease prevalence Plant pathogen*Puccinia psidii*Sensitivity Specificity

## Notes

### Acknowledgments

This study was supported by the Australian Centre of Excellence for Risk Analysis (now named the Centre of Excellence for Biosecurity Risk Analysis). We sincerely thank the experts who participated in the elicitation of priors, the biosecurity officers and volunteers who facilitated the collection of samples, and the scientists who performed the laboratory testing. We are also very grateful to Andrew Robinson, Bonnie Wintle, Marissa McBride and Mark Burgman for their advice. Finally, we appreciate the time taken by two anonymous reviewers for providing detailed comments which have greatly improved the manuscript.

### References

- Andersen S (1997) Re: “Bayesian estimation of disease prevalence and the parameters of diagnostic tests in the absence of a gold standard”. American Journal of Epidemiology 145:290–291Google Scholar
- Aven T (2013) On the meaning and use of the risk appetite concept. Risk Analysis 33:462–468Google Scholar
- Beavers DP, Stamey JD (2012) Bayesian sample size determination for binary regression with a misclassified covariate and no gold standard. Computation Statistics and Data Analysis 56:2574–2582Google Scholar
- Black MA, Craig BA (2002) Estimating disease prevalence in the absence of a gold standard. Statistics in Medicine 21:2653–2669Google Scholar
- Burgman MA, McBride M, Ashton R, Speirs-Bridge A, Flander L, Wintle B, Fidler F, Rumpff L, Twardy C (2011) Expert status and performance. PLoS ONE 6:e22,998Google Scholar
- Carnegie AJ, Lidbetter JR, Walker J, Horwood MA, Tesoriero L, Glen M, Priest MJ (2010)
*Uredo rangelii*, a taxon in the guava rust complex, newly recorded on Myrtaceae in Australia. Australasian Plant Pathology 39:463–466Google Scholar - Casella G, George EI (1992) Explaining the Gibbs Sampler. The American Statistician 46:167–174Google Scholar
- Coutinho TA, Wingfield MJ, Alfenas AC, Crous PW (1998) Eucalyptus rust: a disease with the potential for serious international implications. Plant Disease 82:819–825Google Scholar
- Demissie K, White N, Joseph L, Ernst P (1998) Bayesian estimation of asthma prevalence, and comparison of exercise and questionnaire diagnostics in the absence of a gold standard. Annals of Epidemiology 8:201–208Google Scholar
- Dendukuri N, Joseph L (2001) Bayesian approaches to modeling the conditional dependence between multiple diagnostic tests. Biometrics 57:158–167Google Scholar
- Dongen SV (2006) Prior specification in Bayesian statistics: three cautionary tales. Journal of Theoretical Biology 242:90–100Google Scholar
- Dorny P, Phiri IK, Vercruysse J, Gabriel S, Willingham AL, Brandt J, Victor B, Speybroeck N, Berkvens D (2004) A Bayesian approach for estimating values for prevalence and diagnostic test characteristics of porcine cysticercosis. International Journal for Parasitology 34:569–576Google Scholar
- Dunson DB (2001) Commentary: practical advantages of Bayesian analysis of epidemiologic data. American Journal of Epidemiology 153:1222–1226Google Scholar
- Enoe C, Georgiadis MP, Johnson WO (2000) Estimation of sensitivity and specificity of diagnostic tests and disease prevalence when the true disease state is unknown. Preventive Veterinary Medicine 45:61–81Google Scholar
- Epstein LD, Munoz A, He D (1996) Bayesian imputation of predictive values when covariate information is available and gold standard diagnosis is unavailable. Statistics in Medicine 15:463–476Google Scholar
- Ferson S, Ginzburg LR (1996) Different methods are needed to propagate ignorance and variability. Reliability Engineering & System Safety 54(2):133–144Google Scholar
- Garber A, Solomon N (1999) Cost-effectiveness of alternative test strategies for the diagnosis of coronary artery disease. Annals of Internal Medicine 130:719–728Google Scholar
- Geisser S, Johnson W (1992) Optimal administration of dual screening tests for detecting a characteristic with special reference to low prevalence diseases. Biometrics 48:839–852Google Scholar
- Georgiadis MP, Johnson WO, Gardner IA, Singh R (2003) Correlation-adjusted estimation of sensitivity and specificity of two diagnostic tests. Journal of the Royal Statistical Society: Series C (Applied Statistics) 52(1):63–76Google Scholar
- Glen M, Alfenas AC, Zauza EAV, Wingfield MJ, Mohammed C (2007)
*Puccinia psidii*: a threat to the Australian environment and economy - a review. Australasian Plant Pathology 36:1–16Google Scholar - Grgurinovic CA, Walsh D, Macbeth F (2006) Eucalyptus rust caused by
*Puccinia psidii*and the threat it poses to Australia. EPPO Bulletin 36:486–489Google Scholar - de Groot JAH, Dendukuri N, Janssen KJM, Reitsma JB, Brophy J, Joseph L, Bossuyt PMM, Moons KGM (2012) Adjusting for partial verification or workup bias in meta-analyses of diagnostic accuracy studies. American Journal of Epidemiology 174:847–853Google Scholar
- Hanson T, Johnson WO, Gardner IA (2003) Hierarchical models for estimating herd prevalence and test accuracy in the absence of a gold standard. Journal of agricultural, biological, and environmental statistics 8(2):223–239Google Scholar
- Holliday JL, Jones SA, Simpson JA, Glen M, Edwards J, Robinson A, Burgman MA (2013) A novel spore collection device for sampling exposure pathways: a case study of
*Puccinia psidii*. Plant Disease 97:828–834Google Scholar - Johnson WO, Gastwirth JL, Pearson LM (2001) Screening without a “Gold Standard”: the Hui-Walter paradigm revisited. American Journal of Epidemiology 153:921–924Google Scholar
- Joseph L (1997) Re: “Bayesian estimation of disease prevalence and the parameters of diagnostic tests in the absence of a gold standard” The first author replies. American Journal of Epidemiology 145:291Google Scholar
- Joseph L, Gyorkos TW, Coupal L (1995) Bayesian estimation of disease prevalence and the parameters of diagnostic tests in the absence of a gold standard. American Journal of Epidemiology 141:263–272Google Scholar
- Kawanishi T, Uematsu S, Kakishima M, Kagiwada S, Hamamoto H, Horie H, Namba S (2009) First report of rust disease on ohia and the causal fungus,
*Puccinia psidii*, in Japan. Journal of General Plant Pathology 75:428–431Google Scholar - Keefer DL, Bodily SE (1983) Three-point approximations for continuous random variables. Management Science 29:595–609Google Scholar
- Krull CR, Waipara NW, Choquenot D, Burns BR, Gormley AM, Stanley MC (2013) Absence of evidence is not evidence of absence: feral pigs as vectors of soil-borne pathogens. Austral Ecology 38:534–542Google Scholar
- Langrell SRH, Glen M, Alfenas AC (2008) Molecular diagnosis of
*Puccinia psidii*(guava rust) - a quarantine threat to Australian eucalypt and Myrtaceae biodiversity. Plant Pathology 57:687–701Google Scholar - Lew RA, Levy PS (1989) Estimation of prevalence on the basis of screening tests. Statistics in Medicine 8(10):1225–1230Google Scholar
- Lewis F, Sanchez-Vanquez MJ, Torgerson PR (2012) Association between covariates and disease occurrence in the presence of diagnostic error. Epidemiology and Infection 140:1515–1524Google Scholar
- Liu J, Chen F, Yu H, Zeng P, Liu L (2014) A two-stage Bayesian method for estimating accuracy and disease prevalence for two dependent dichotomous screening tests when the status of individuals who are negative on both tests is unverified. BMC medical research methodology 14(1):110Google Scholar
- Lu Y, Dendukuri N, Schiller I, Joseph L (2010) A Bayesian approach to simultaneously adjusting for verification and reference standard bias in diagnostic test studies. Statistics in Medicine 29:2532–2543Google Scholar
- Makowski D, Denis J-B, Ruck L, Penaud A (2008) A Bayesian approach to assess the accuracy of a diagnostic test based on plant disease measurement. Crop Protection 27:1187–1193Google Scholar
- Martin TG, Kuhnert PM, Mengersen K, Possingham HP (2005) The power of expert opinion in ecological models using Bayesian methods: impact of grazing on birds. Journal of Applied Ecology 15:266–280Google Scholar
- McBride MF, Fidler F, Burgman MA (2012) Evaluating the accuracy and calibration of expert predictions under uncertainty: predicting the outcomes of ecological research. Diversity and Distributions 18:782–794Google Scholar
- Mendoza-Blanco JR, Tu XM, Iyengar S (1996) A Bayesian inference on prevalence using a missing-data approach with simulation-based techniques: applications to HIV screening. Statistics in Medicine 15:2161–2176Google Scholar
- Mila AL, Yang XB, Carriquiry AL (2003) Bayesian logistic regression of soybean Sclerotinia stem rot prevalence in the U.S. north central region: accounting for uncertainty in parameter estimation. Phytopathology 93:758–764Google Scholar
- Mireku E, Simpson JA (2002) Fungal and nematode threats to Australian forests and amenity trees from importation of wood and wood products. Canadian Journal of Plant Pathology 24:117–124Google Scholar
- Morris WK, Vesk PA, McCarthy MA (2013) Profiting from pilot studies: analysing mortality using Bayesian models with informative priors. Basic and Applied Ecology 14:81–89Google Scholar
- Nauta MJ (2000) Separation of uncertainty and variability in quantitative microbial risk assessment models. International Journal of Food Microbiology 57(1):9–18Google Scholar
- Pennello GA (2011) Bayesian analysis of diagnostic test accuracy when disease state is unverified for some subjects. Journal of Biopharmaceutical Statistics 21:954–970Google Scholar
- Pollino CA, White AK, Hart BT (2007) Examination of conflicts and improved strategies for the management of an endangered Eucalypt species using Bayesian networks. Ecological Modelling 201:37–59Google Scholar
- Poole WK, Flynn PM, Rao AV, Cooley PC (1996) Estimating the prevalence of drug use from self-reports in a cohort for which biologic data are available for a subsample. American Journal of Epidemiology 144:413–420Google Scholar
- Rahme E, Joseph L (1998) Estimating the prevalence of a rare disease: adjusted maximum likelihood. Journal of the Royal Statistical Society: Series D (The Statistician) 47(1):149–158Google Scholar
- Reitsma JB, Rutjes AW, Khan KS, Coomarasamy A, Bossuyt PM (2009) A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard. Journal of clinical epidemiology 62(8):797–806Google Scholar
- Rubin DB (1988) Using the SIR algorithm to simulate posterior distributions. Bayesian statistics 3(1):395–402Google Scholar
- Schiffman M, Wheeler CM, Dasgupta A, Solomon D, Castle PE (2005) A comparison of a prototype PCR assay and Hybrid Capture 2 for detection of carcinogenic human papillomavirus DNA in women with equivocal or mildly abnormal papanicolaou smears. American Journal of Clinical Pathology 124:722–732Google Scholar
- Scott AN, Joseph L, Belisle P, Behr MA, Schwartzman K (2008) Bayesian modelling of tuberculosis clustering from DNA fingerprint data. Statistics in Medicine 27:140–156Google Scholar
- Simpson JA, Thomas K, Grgurinovic CA (2006) Uredinales species pathogenic on species of Myrtaceae. Australasian Plant Patholology 35:549–562Google Scholar
- Speirs-Bridge A, Fidler F, McBride M, Flander L, Cumming G, Burgman M (2010) Reducing overconfidence in the interval judgments of experts. Risk Analysis 30:512–523Google Scholar
- Spiegelhalter DJ, Best NG (2003) Bayesian approaches to multiple sources of evidence and uncertainty in complex cost-effectiveness modelling. Statistics in Medicine 22(23):3687–3709Google Scholar
- Stubner S (2002) Enumeration of 16S rDNA of Desulfotomaculum lineage 1 in rice field soil by real-time PCR with SybrGreen\(_{{\rm TM}}\) detection. Journal of Microbiological Methods 50:155–164Google Scholar
- Toft N, Jørgensen E, (2005) Diagnosing diagnostic tests: evaluating the assumptions underlying the estimation of sensitivity and specificity in the absence of a gold standard. Preventive veterinary medicine 68(1):19–33Google Scholar
- Tommerup IC, Alfenas AC, Old KM (2003) Guava rust in Brazil - a threat to eucalyptus and other Myrtaceae. New Zealand Journal of Science 33:420–428Google Scholar
- Tu XM, Kowalski J, Jia G (1999) Bayesian analysis of prevalence with covariates using simulation-based techniques: applications to HIV screening. Statistics in Medicine 18:3059–3073Google Scholar
- Vettraino AM, Sukno S, Vannini A, Garbelotto M (2010) Diagnostic sensitivity and specificity of different methods used by two laboratories for the detection of
*Phytophthora ramorum*on multiple natural hosts. Plant Pathology 59:289–300Google Scholar - Vidal E, Moreno A, Berolini E, Cambra M (2012) Estimation of the accuracy of two diagnostic methods for the detection of Plum pox virus in nursery blocks by latent class models. Plant Pathology 61:413–422Google Scholar
- Vijan S, Hwang EW, Hofer TP, Hayward RA (2001) Which colon cancer screening test? a comparison of costs, effectiveness, and compliance. American Journal of Medicine 111:593–601Google Scholar
- Vlems FA, Ladanyi A, Gertler R, Rosenberg R, Diepstra JHS, Rder C, Nekarda H, Molnar B, Tulassay Z, van Muijen GNP, Vogel I (2003) Reliability of quantitative reverse-transcriptase-PCR-based detection of tumour cells in the blood between different laboratories using a standardised protocol. European Journal of Cancer 39:388–396Google Scholar