European Journal of Plant Pathology

, Volume 139, Issue 1, pp 79–94 | Cite as

Comparison of statistical models in a meta-analysis of fungicide treatments for the control of citrus black spot caused by Phyllosticta citricarpa

  • D. Makowski
  • A. Vicent
  • M. Pautasso
  • G. Stancanelli
  • T. Rafoss


Meta-analysis has been recognised as a powerful method to synthetize existing published data from different studies through a formal statistical analysis. Several statistical models have been proposed to evaluate the effectiveness of treatments against plant diseases using meta-analysis, but the sensitivity of the estimated treatment effects to the model chosen has not been investigated in detail in the context of plant pathology. In this paper, four different statistical models were defined to analyse fungicide control trials with binary outcomes. These models were used to conduct a meta-analysis on the effectiveness of fungicide treatments against citrus black spot, a fungal disease caused by the quarantine pathogen Phyllosticta citricarpa. The models differed in the assumption made on the variability of the treatment effect (constant or variable between experimental plots) and in the method used for parameter estimation (classical or Bayesian). Odds ratios were estimated for two groups of fungicides, copper compounds and dithiocarbamates, widely applied for CBS control using each model in turn. Classical and Bayesian statistical models led to similar results, but the estimated treatment effectiveness and their associated levels of uncertainty were sensitive to the assumption made about the variability of the treatment effect. Estimated odds ratios were different depending on whether the treatment effect was assumed to be constant or variable between experimental plots. The size of the confidence intervals was underestimated when the treatment effect was assumed constant while it was variable in reality. Because of the strong between-plot variability, the 90 % percentiles of the odds ratios were much higher than the point estimates, and this result revealed that, in some plots, treatment effectiveness could be much lower than expected. Based on our results, we conclude that it is not sufficient to calculate point estimates of odds ratio when the between-plot variability of the treatment effect is strong and that, in such case, it is recommended to compute the predictive distributions of the odds ratio.


Bayesian Categorical data Disease management Generalized mixed-effect linear model Meta-analysis Odds ratio Guignardia citricarpa Sensitivity analysis 



We thank C. Llacer for help in data extraction to generate the database used in this study. AV was supported by the program DOC-INIA.


The present paper is published under the sole responsibility of the authors and may not be considered as a scientific output by the European Food Safety Authority (EFSA). The position and opinions presented in this publication are those of the authors alone and do not necessarily represent the views of EFSA.


  1. Agresti, A. (2002). Categorical data analysis (2nd ed.). Hoboken: Wiley.CrossRefGoogle Scholar
  2. Baldassari, R. B., Wickert, E., & de Goes, A. (2008). Pathogenicity, colony morphology and diversity of isolates of Guignardia citricarpa and G. mangiferae isolated from Citrus spp. European Journal of Plant Pathology, 120, 103–110.CrossRefGoogle Scholar
  3. Ballini, E., Morel, J.-B., Droc, G., Price, A., Courtois, B., Notteghem, J. L., & Tharreau, D. (2008). A genome-wide meta-analysis of rice blast resistance genes and quantitative trait loci provides new insights into partial and complete resistance. Molecular Plant-Microbe Interactions, 21, 859–868.PubMedCrossRefGoogle Scholar
  4. Blankinship, J. C., Niklaus, P. A., & Hungate, B. A. (2011). A meta-analysis of responses of soil biota to global change. Oecologia, 165, 553–565.PubMedCrossRefGoogle Scholar
  5. Borowicz, V. A. (2001). Do arbuscular mycorrhizal fungi alter plant–pathogen relations? Ecology, 82, 3057–3068.Google Scholar
  6. Burnham, K. P., & Anderson, D. R. (2002). Model selection and multimodel inference. New York: Springer.Google Scholar
  7. Carlin, B. P., & Louis, T. A. (2008). Bayesian methods for data analysis (3rd ed., p. 553). Boca Raton: Chapman & Hall/CRC Texts in Statistical Science.Google Scholar
  8. Chaplin-Kramer, R., O’Rourke, M. E., Blitzer, E. J., & Kremen, C. (2011). A meta-analysis of crop pest and natural enemy response to landscape complexity. Ecology Letters, 14, 922–932.PubMedCrossRefGoogle Scholar
  9. de Goes, A., Andrade, A. G., & Moretto, K. C. K. (2000). Efeito de diferentes tipos de óleos na mistura de benomyl + mancozeb no controle de Guignardia citricarpa, agente causal da mancha preta dos frutos cítricos. Summa Phytopathologica, 26, 233–236.Google Scholar
  10. EFSA (European Food Safety Authority). (2010). Application of systematic review methodology to food and feed safety assessments to support decision making. EFSA Journal, 8(6), 1637. doi: 10.2903/j.efsa.2010.1637. Available online: Scholar
  11. EPPO (European and Mediterranean Plant Protection Organization) (2013). PQR-EPPO database on quarantine pests (database version 2013-01-21). Available online at
  12. Er, H. L., Roberts, P. D., Marois, J. J., & van Bruggen, A. H. C. (2013). Potential distribution of citrus black spot in the United States based on climatic conditions. European Journal of Plant Pathology, 137, 635–647.CrossRefGoogle Scholar
  13. Feng, Z., & Kobayashi, K. (2009). Assessing the impacts of current and future concentrations of surface ozone on crop yield with meta-analysis. Atmospheric Environment, 43, 1510–1519.CrossRefGoogle Scholar
  14. Gelman, A., & Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–511.CrossRefGoogle Scholar
  15. Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5, 3–8.CrossRefGoogle Scholar
  16. Hewitt, H. G. (1998). Fungicides in crop protection (p. 221). Wallingford: CAB International.Google Scholar
  17. Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009). A re-evaluation of random-effects meta-analysis. Journal of the Royal Statistical Society A, 172, 137–159.CrossRefGoogle Scholar
  18. Kotze, J. M. (1981). Epidemiology and control of citrus black spot in South Africa. Plant Disease, 65, 945–950.CrossRefGoogle Scholar
  19. Madden, L. V., & Paul, P. A. (2011). Meta-analysis for evidencesynthesis in plant pathology: an overview. Phytopathology, 101, 16–30.PubMedCrossRefGoogle Scholar
  20. Mila, A. L., & Ngugi, H. K. (2011). A Bayesian approach to meta-analysis of plant pathology studies. Phytopathology, 101, 42–51.PubMedCrossRefGoogle Scholar
  21. Ngugi, H. K., Esker, P. D., & Scherm, H. (2011a). Meta-analysis to determine the effects of plant disease management measures: review and case studies on soybean and apple. Phytopathology, 101, 31–41.PubMedCrossRefGoogle Scholar
  22. Ngugi, H. K., Lehman, B. L., & Madden, L. V. (2011b). Multiple treatment meta-analysis of products evaluated for control of Fire Blight in the eastern United States. Phytopathology, 101, 512–522.PubMedCrossRefGoogle Scholar
  23. Normand, S. L. T. (1999). Meta-analysis: formulating, evaluating, combining, and reporting. Statistics in Medicine, 18, 321–359.PubMedCrossRefGoogle Scholar
  24. OECD (Organisation for Economic Co-Operation and Development) (2010). International standards for fruit and vegetables: citrus fruit. OECD, 244 pp.Google Scholar
  25. Ojiambo, P. S., & Scherm, H. (2006). Biological and application-oriented factors influencing plant disease suppression by biological control: a meta-analytical review. Phytopathology, 96, 1168–1174.PubMedCrossRefGoogle Scholar
  26. Ojiambo, P. S., Paul, P. A., & Holmes, G. J. (2010). A quantitative review of fungicide efficacy for managing downy mildew in cucurbits. Phytopathology, 100, 1066–1076.PubMedCrossRefGoogle Scholar
  27. Paul, P. A., Lipps, P. E., & Madden, L. V. (2005). Relationship between visual estimates of Fusarium head blight intensity and deoxynivalenol accumulation in harvested wheat grain: a meta-analysis. Phytopathology, 95, 1225–1236.PubMedCrossRefGoogle Scholar
  28. Paul, P. A., Lipps, P. E., Hershman, D. E., McMullen, M. P., Draper, M. A., & Madden, L. V. (2008). Efficacy of triazole-based fungicides for Fusarium head blight and desoxynivalenol control in wheat: a meta-analysis. Phytopathology, 98, 999–1011.PubMedCrossRefGoogle Scholar
  29. Philibert, A., Loyce, C., & Makowski, D. (2012). Assessment of the quality of the meta-analysis in agronomy. Agriculture, Ecosystems and Environment, 148, 72–82.CrossRefGoogle Scholar
  30. Rosenberg, M. S., Garrett, K. A., Su, Z., & Bowden, R. L. (2004). Meta-analysis in plant pathology: synthesizing research results. Phytopathology, 94, 1013–1017.PubMedCrossRefGoogle Scholar
  31. Salam, K. P., Thomas, G. J., Beard, C., Loughman, R., MacLeod, W. J., & Salam, M. U. (2013). Application of meta-analysis in plant pathology: a case study examining the impact of fungicides on wheat yield loss from the yellow spot—septoria nodorum blotch disease complex in Western Australia. Food Security, 5, 319–325.CrossRefGoogle Scholar
  32. Scheepmaker, J. W. A., & van de Kassteele, J. (2011). Effects of chemical control agents and microbial biocontrol agents on numbers of non-target microbial soil organisms: a meta-analysis. Biocontrol Science and Technology, 21, 1225–1242.CrossRefGoogle Scholar
  33. Scherm, H., Christiano, R. S. C., Esker, P. D., Del Ponte, E. M., & Godoy, C. V. (2009). Quantitative review of fungicide efficacy trials for managing soybean rust in Brazil. Crop Protection, 28, 774–782.CrossRefGoogle Scholar
  34. Schutte, G. C., Beeton, K. V., & Kotzé, J. M. (1997). Rind stippling on Valencia oranges by copper fungicides used for control of citrus black spot in South Africa. Plant Disease, 81, 851–854.CrossRefGoogle Scholar
  35. Schutte, G. C., Mansfield, R. I., Smith, H., & Beeton, K. V. (2003). Application of azoxystrobin for control of benomyl-resistant Guignardia citricarpa on ‘Valencia’ oranges in South Africa. Plant Disease, 87, 784–788.CrossRefGoogle Scholar
  36. Song, F., Altman, D. G., Glenny, A. M., & Deeks, J. J. (2003). Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ, 326, 472.PubMedCentralPubMedCrossRefGoogle Scholar
  37. Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & van der Linde, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society B, 64, 583–639.CrossRefGoogle Scholar
  38. Spósito, M. B., Amorim, L., Bassanezi, R. B., Yamamoto, P., Felippe, M. R., & Czermainski, A. B. C. (2011). Relative importance of inoculum sources of Guignardia citricarpa on the citrus black spot epidemic in Brazil. Crop Protection, 30, 1546–1552.CrossRefGoogle Scholar
  39. Stijnen, T., Hamza, T. H., & Özdemir, P. (2010). Random effects meta-analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data. Statistics in Medicine, 29, 3046–3067.PubMedCrossRefGoogle Scholar
  40. Sutton, A. J., Abrams, K. R., Jones, D. R., Sheldon, T. A., & Song, F. (2000). Methods for meta-analysis in medical research. Chichester: Wiley.Google Scholar
  41. van de Wouw, M., van Hintum, T., Kik, C., van Treuren, R., & Visser, B. (2010). Genetic diversity trends in twentieth century crop cultivars: a meta analysis. Theoretical and Applied Genetics, 120, 1241–1252.PubMedCentralPubMedCrossRefGoogle Scholar
  42. Warn, D. E., Thompson, S. G., & Spiegelhalter, D. J. (2002). Bayesian random effects meta-analysis of trials with binary outcomes: methods for the absolute risk difference and relative risk scales. Statistics in Medicine, 21, 1601–1623.PubMedCrossRefGoogle Scholar
  43. Wikee, S., Udayanga, D., Crous, P. W., Chukeatirote, E., McKenzie, E. H. C., Bahkali, A. H., Dai, D. Q., & Hyde, K. D. (2011). Phyllosticta—an overview of current status of species recognition. Fungal Diversity, 51, 43–61.CrossRefGoogle Scholar

Copyright information

© KNPV 2014

Authors and Affiliations

  • D. Makowski
    • 1
  • A. Vicent
    • 2
  • M. Pautasso
    • 3
  • G. Stancanelli
    • 4
  • T. Rafoss
    • 5
  1. 1.INRA, UMR 211 INRA AgroParisTechThiverval-GrignonFrance
  2. 2.Instituto Valenciano de Investigaciones Agrarias (IVIA)MoncadaSpain
  3. 3.Forest Pathology and Dendrology, Institute of Integrative BiologyETH ZurichZurichSwitzerland
  4. 4.Plant Health UnitEuropean Food Safety AuthorityParmaItaly
  5. 5.Plant Health & Plant Protection DivisionBioforskÅsNorway

Personalised recommendations