Bias Correction in Estimating Proportions by Pooled Testing

  • Graham HepworthEmail author
  • Brad J. Biggerstaff


In the estimation of proportions by pooled testing, the MLE is biased, and several methods of correcting the bias have been presented in previous studies. We propose a new estimator based on the bias correction method introduced by Firth (Biometrika 80:27–38, 1993), which uses a modification of the score function, and we provide an easily computable, Newton–Raphson iterative formula for its computation. Our proposed estimator is almost unbiased across a range of problems, and superior to existing methods. We show that for equal pool sizes the new estimator is equivalent to the estimator proposed by Burrows (Phytopathology 77:363–365, 1987). The performance of our estimator is examined using pooled testing problems encountered in plant disease assessment and prevalence estimation of mosquito-borne viruses.

Supplementary materials accompanying this paper appear online.


Bias correction Estimation of proportions Group testing Pooled testing Virus prevalence 

Supplementary material

13253_2017_297_MOESM1_ESM.pdf (155 kb)
Supplementary material 1 (pdf 155 KB)


  1. Aranda, C., Sanchez-Seco, M. P., Caceres, F., Escosa, R., Galvez, J. C., Masia, M., Marques, E., Ruz, S., Alba, A., Busquets, N., Vazquez, A., Castella, J., and Tenorio, A. (2009). Detection and monitoring of mosquito flaviviruses in Spain between 2001 and 2005. (2009), Vector-borne and Zoonotic Diseases 9, doi: 10.1089/vbz.2008.0073.
  2. Bilder, C. R. and Tebbs, J. M. (2005). Empirical Bayes estimation of the disease transmission probability in multiple-vector-transfer designs. Biometrical Journal 47, 502–516.MathSciNetCrossRefGoogle Scholar
  3. Brookmeyer, R. (1999). Analysis of multistage pooling studies of biological specimens for estimating disease incidence and prevalence. Biometrics 55, 608–612.Google Scholar
  4. Burrows, P. M. (1987). Improved estimation of pathogen transmission rates by group testing. Phytopathology 77 363–365.CrossRefGoogle Scholar
  5. Colon, S., Patil, G. P., and Taillie, C. (2001) Estimating prevalence using composites. Environmental and Ecological Statistics 8, 213–236.MathSciNetCrossRefGoogle Scholar
  6. Ding, J. and Xiong, W. (2016). A new estimator for a population proportion using group testing. Communications in Statistics–Simulation and Computation 45, doi: 10.1080/03610918.2013.854909.
  7. Dorfman, R. (1943). The detection of defective members of large populations.Annals of Mathematical Statistics 14, 436–440.CrossRefGoogle Scholar
  8. Durnez, L., Eddyani, M., Mgode, G. F., Katakweba, A., Katholi, C. R., Machang’u, R. R., Kazwala, R. R., and Portaeis, F. and Leirs, H. (2008). First detection of mycobacteria in African rodents and insectivores, using stratified pool screening. Applied and Environmental Microbiology 74, 768–773.CrossRefGoogle Scholar
  9. Firth, D. (1993). Bias reduction of maximum likelihood estimates. Biometrika 80, 27–38.MathSciNetCrossRefzbMATHGoogle Scholar
  10. Freeman, A. J., Spackman, M. E., Aftab, M., McQueen, V., King, S., van Leur, J. A., Loh, M. H., and Rodoni, B. (2013). Comparison of tissue blot immunoassay and reverse transcription polymerase chain reaction assay for virus-testing pulse crops from a South-Eastern Australia survey. Australasian Plant Pathology 42, 675–683.CrossRefGoogle Scholar
  11. Gart, J. J. (1991). An application of score methodology: Confidence intervals and tests of fit for one-hit curves. In Handbook of Statistics, C. R. Rao, R. Chakraborty (eds), 8, 395–406. Amsterdam: Elsevier.Google Scholar
  12. Gildow, F. E., Shah, D. A., Sackett, W. M., Butzler, T., Nault, B. A., and Fleischer, S. J. (2008). Transmission efficiency of Cucumber mosaic virus by aphids associated with virus epidemics in snap bean. Phytopathology 98, 1233–1241.CrossRefGoogle Scholar
  13. Godsey, M. S., Nasci, R., Savage, H. M., Aspen, S., King, R., Powers, A. M., Burkhalter, K., Colton, L., Charnetzky, D., Lasater, S., Taylor, V., and Palmisano, C. T. (2005). West Nile Virus-infected mosquitoes, Louisiana, 2002. Emerging Infectious Diseases 11, 1399–1404.CrossRefGoogle Scholar
  14. Heinze, G. and Schemper, M. (2002). A solution to the problem of separation in logistic regression. Statistics in Medicine 21, 2409–2419.CrossRefGoogle Scholar
  15. Hepworth, G. (1996). Exact confidence intervals for proportions estimated by group testing. Biometrics 52, 1134–1146.MathSciNetCrossRefzbMATHGoogle Scholar
  16. Hepworth, G. (2005). Confidence intervals for proportions estimated by group testing with groups of unequal size. JABES 10, 478–497.CrossRefGoogle Scholar
  17. Hepworth, G. and Watson, R. (2009). Debiased estimation of proportions in group testing. JRSS-C 58, 105–121.MathSciNetGoogle Scholar
  18. Hund, L. and Pagano, M. (2013). Estimating HIV prevalence from surveys with low individual consent rates: annealing individual and pooled samples. Emerging Themes in Epidemiology 10:2.CrossRefGoogle Scholar
  19. Liu, S-C., Chiang, K-S., Lin, C-H., and Deng, T-C. (2011). Confidence interval procedures for proportions estimated by group testing with groups of unequal size adjusted for overdispersion. Journal of Applied Statistics 38, 1467–1482.MathSciNetCrossRefzbMATHGoogle Scholar
  20. Mehrabi, Y. and Matthews, J. N. (1995). Likelihood-based methods for bias reduction in limiting dilution assays. Biometrics 51, 1543–1549.CrossRefzbMATHGoogle Scholar
  21. Mitchell, S. and Pagano, M. (2012). Pooled testing for effective estimation of the prevalence of Schistosoma mansoni. American Journal of Tropical Medicine and Hygiene 87, 850–861.CrossRefGoogle Scholar
  22. Schaarschmidt, F. (2007). Experimental design for one-sided confidence intervals or hypothesis tests in binomial group testing. Communications in Biometry and Crop Science 2, 32–40.Google Scholar
  23. Swallow, W. H. (1985). Group testing for estimating infection rates and probabilities of disease transmission. Phytopathology 75, 882–889.CrossRefGoogle Scholar
  24. Thompson, K. H. (1962). Estimation of the proportion of vectors in a natural population of insects. Biometrics 18, 568–578.CrossRefGoogle Scholar
  25. Walter, S. D., Hildreth, S. W. and Beaty, B.J. (1980). Estimation of infection rates in populations of organisms using pools of variable size. American Journal of Epidemiology 112, 124–128.CrossRefGoogle Scholar
  26. Yamamura, K. and Hino, A. (2007). Estimation of the proportion of defective units by using group testing under the existence of a threshold of detection. Communications in Statistics–Simulation and Computation 36, 949–957.MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© International Biometric Society 2017

Authors and Affiliations

  1. 1.School of Mathematics and StatisticsThe University of MelbourneVictoriaAustralia
  2. 2.Centers for Disease Control and PreventionFort CollinsUSA

Personalised recommendations