Skip to main content

Multiple Comparisons for Multiple Endpoints in Agricultural Experiments

Abstract

Agricultural experiments often have a completely randomized design, and multiple, correlated variables are measured. This paper addresses an appropriate statistical evaluation. A multivariate t-distribution is used for the calculation of multiplicity-adjusted p-values and simultaneous confidence intervals. The number of the multiple variables as well as their correlations are taken into account this way. We consider ratios of means instead of differences, and comparisons versus the overall mean instead of all-pair comparisons. A data set from a greenhouse experiment with glucosinolates of several cultivars of Chinese cabbage (Brassica rapa subsp. pekinensis) is used as an example. Related code based on the R-package SimComp is presented. This package allows a wide application in many agricultural experiments with a similar design.

This is a preview of subscription content, access via your institution.

References

  1. Bandyopadhyay, S., Ganguli, B., and Chatterjee, A. (2011), “A Review of Multivariate Longitudinal Data Analysis,” Statistical Methods in Medical Research, 20 (4), 299–330.

    MathSciNet  Article  MATH  Google Scholar 

  2. Bretz, F. (2006), “An Extension of the Williams Trend Test to General Unbalanced Linear Models,” Computational Statistics & Data Analysis, 50 (7), 1735–1748.

    MathSciNet  Article  MATH  Google Scholar 

  3. Bretz, F., Genz, A., and Hothorn, L. A. (2001), “On the Numerical Availability of Multiple Comparison Procedures,” Biometrical Journal, 43, 645–656.

    MathSciNet  Article  MATH  Google Scholar 

  4. Buonaccorsi, J. P., and Iyer, H. K. (1984), “A Comparison of Confidence Regions and Designs in Estimation of a Ratio,” Communications in Statistics. Theory and Methods, 13, 723–741.

    MATH  Google Scholar 

  5. Dilba, G., Bretz, F., Guiard, V., and Hothorn, L. A. (2004), “Simultaneous Confidence Intervals for Ratios with Applications to the Comparison of Several Treatments with a Control,” Methods of Information in Medicine, 43 (5), 465–469.

    Google Scholar 

  6. Djira, G. D., and Hothorn, L. A. (2009), “Detecting Relative Changes in Multiple Comparisons with an Overall Mean,” Journal of Quality Technology, 41, 60–65.

    Google Scholar 

  7. Dunnett, C. W. (1955), “A Multiple Comparison Procedure for Comparing Several Treatments with a Control,” Journal of the American Statistical Association, 50 (272), 1096–1121.

    Article  MATH  Google Scholar 

  8. Faes, C., Molenberghs, G., Aerts, M., Verbeke, G., and Kenward, M. G. (2009), “The Effective Sample Size and an Alternative Small-Sample Degrees-of-Freedom Method,” American Statistician, 63 (4), 389–399.

    MathSciNet  Article  MATH  Google Scholar 

  9. Fieller, E. C. (1954), “Some Problems in Interval Estimation,” Journal of the Royal Statistical Society, Series B, 16, 175–185.

    MathSciNet  MATH  Google Scholar 

  10. Fieuws, S., Verbeke, G., and Mollenberghs, G. (2007), “Random-Effects Models for Multivariate Repeated Measures,” Statistical Methods in Medical Research, 16 (5), 387–397.

    MathSciNet  Article  Google Scholar 

  11. Frömke, C., and Bretz, F. (2004), “Simultaneous Tests and Confidence Intervals for the Evaluation of Agricultural Field Trials,” Agronomy Journal, 96 (5), 1323–1330.

    Article  Google Scholar 

  12. Genz, A., and Bretz, F. (2002), “Methods for the Computation of Multivariate T-Probabilities,” Journal of Computational and Graphical Statistics, 11, 950–971.

    MathSciNet  Article  Google Scholar 

  13. Genz, A., Bretz, F., Miwa, T., Mi, X., Leisch, F., Scheipl, F., and Hothorn, T. (2012), mvtnorm: Multivariate Normal and t Distributions. R package version 0.9-9994, available at http://CRAN.R-project.org/package=mvtnorm.

  14. Gerendas, J., Breuning, S., Stahl, T., Mersch-Sundermann, V., and Mühling, K. H. (2008), “Isothiocyanate Concentration in Kohlrabi (Brassica Oleracea L. Var. Gongylodes) Plants as Influenced by Sulfur and Nitrogen Supply,” Journal of Agricultural and Food Chemistry, 56 (18), 8334–8342.

    Article  Google Scholar 

  15. Guilbaud, O. (2011), “Note on Simultaneous Inferences About Non-Inferiority and Superiority for a Primary and a Secondary Endpoint,” Biometrical Journal, 53, 6 (SI), 927–937.

    MathSciNet  Article  Google Scholar 

  16. Hasler, M. (2012), SimComp: Simultaneous Comparisons for Multiple Endpoints. R package version 1.7.0, available at http://CRAN.R-project.org/package=SimComp.

  17. Hasler, M., and Hothorn, L. A. (2008), “Multiple Contrast Tests in the Presence of Heteroscedasticity,” Biometrical Journal, 50 (5), 793–800.

    MathSciNet  Article  Google Scholar 

  18. — (2011), “A Dunnett-Type Procedure for Multiple Endpoints,” The International Journal of Biostatistics, 7 (1), 3.

    MathSciNet  Article  Google Scholar 

  19. — (2012), “A Multivariate Williams-Type Trend Procedure,” Statistics in Biopharmaceutical Research, 4, 57–65.

    Article  Google Scholar 

  20. Hochberg, Y. (1988), “A Sharper Bonferroni Procedure for Multiple Tests of Significance,” Biometrika, 75 (4), 800–802.

    MathSciNet  Article  MATH  Google Scholar 

  21. Holm, S. (1979), “A Simple Sequentially Rejective Multiple Test Procedure,” Scandinavian Journal of Statistics, 6, 65–70.

    MathSciNet  MATH  Google Scholar 

  22. Hommel, G. (1988), “A Stagewise Rejective Multiple Test Procedure Based on a Modified Bonferroni Test,” Biometrika, 75 (2), 383–386.

    Article  MATH  Google Scholar 

  23. Hothorn, T., Bretz, F., and Genz, A. (2001), “On Multivariate t and Gauss Probabilities in R,” R News, 1 (2), 27–29.

    Google Scholar 

  24. Hsu, J. (1996), “Multiple Comparisons: Theory and Methods,” 1 edn., London: Chapman & Hall/CRC.

    Book  MATH  Google Scholar 

  25. Huang, P., Tilley, B. C., Woolson, R. F., and Lipsitz, S. (2005), “Adjusting O’Brien’s Test to Control Type i Error for the Generalized Nonparametric Behrens-Fisher Problem,” Biometrics, 61 (2), 532–539.

    MathSciNet  Article  MATH  Google Scholar 

  26. ICH E9 Expert Working Group (1999), “ICH Harmonised Tripartite Guideline: Statistical Principles for Clinical Trials,” Statistics in Medicine, 18 (15), 1905–1942.

    Google Scholar 

  27. Konietschke, F., Hothorn, L. A., and Brunner, E. (2012), “Rank-Based Multiple Test Procedures and Simultaneous Confidence Intervals,” Electronic Journal of Statistics, 6, 738–759.

    MathSciNet  Article  MATH  Google Scholar 

  28. Laird, N. M., and Ware, J. H. (1982), “Random-Effects Models for Longitudinal Data,” Biometrics, 38 (4), 963–974.

    Article  MATH  Google Scholar 

  29. Läuter, J., Glimm, E., and Kropf, S. (1996), “New Multivariate Tests for Data with an Inherent Structure,” Biometrical Journal, 38 (1), 5–23.

    MathSciNet  Article  MATH  Google Scholar 

  30. — (1998), “Multivariate Tests Based on Left-Spherically Distributed Linear Scores,” The Annals of Statistics, 26 (5), 1972–1988.

    MathSciNet  Article  MATH  Google Scholar 

  31. Liu, Y., Hsu, J., and Ruberg, S. (2007), “Partition Testing in Dose-Response Studies with Multiple Endpoints,” Pharmaceutical Statistics, 6 (3), 181–192.

    Article  Google Scholar 

  32. Liu, W., Ah-Kine, P., Bretz, F., and Hayter, A. J. (2013), “Exact Simultaneous Confidence Intervals for a Finite Set of Contrasts of Three, Four or Five Generally Correlated Normal Means,” Computational Statistics & Data Analysis, 57, 141–148.

    MathSciNet  Article  Google Scholar 

  33. Mithen, R. F., Dekker, M., Verkerk, R., Rabot, S., and Johnson, I. T. (2000), “The Nutritional Significance, Biosynthesis and Bioavailability of Glucosinolates in Human Foods,” Journal of the Science of Food and Agriculture, 80 (7), 967–984.

    Article  Google Scholar 

  34. Nelson, P. R. (1989), “Multiple Comparisons of Means Using Simultaneous Confidence Intervals,” Journal of Quality Technology, 21 (4), 232–241.

    Google Scholar 

  35. Neuhäuser, M. (2006), “How to Deal with Multiple Endpoints in Clinical Trials,” Fundamental & Clinical Pharmacology, 20 (6), 515–523.

    Article  Google Scholar 

  36. R Core Team (2012), R: a Language and Environment for Statistical Computing, Vienna: R Foundation for Statistical Computing. ISBN 3-900051-07-0. URL: http://www.R-project.org/.

    Google Scholar 

  37. Rangkadilok, N., Nicolas, M. E., Bennett, R. N., Eagling, D. R., Premier, R. R., and Taylor, P. W. J. (2004), “The Effect of Sulfur Fertilizer on Glucoraphanin Levels in Broccoli (B. Oleracea L. Var. Italica) at Different Growth Stages,” Journal of Agricultural and Food Chemistry, 52 (9), 2632–2639.

    Article  Google Scholar 

  38. Schaarschmidt, F., and Vaas, L. (2009), “Analysis of Trials with Complex Treatment Structure Using Multiple Contrast Tests,” HortScience, 44 (1), 188–195.

    Google Scholar 

  39. Schonhof, I., Blankenburg, D., Müller, S., and Krumbein, A. (2007), “Sulfur and Nitrogen Supply Influence Growth, Product Appearance, and Glucosinolate Concentration of Broccoli,” Journal of Plant Nutrition and Soil Science, 170 (1), 65–72.

    Article  Google Scholar 

  40. Strassburger, K., and Bretz, F. (2008), “Compatible Simultaneous Lower Confidence Bounds for the Holm Procedure and Other Bonferroni-Based Closed Tests,” Statistics in Medicine, 27 (24), 4914–4927.

    MathSciNet  Article  Google Scholar 

  41. The MathWorks Inc. (2010), MATLAB, Natick, Massachusetts. Version 7.10.0 (R2010a).

  42. Tukey, J. W. (1953), The Problem of Multiple Comparisons Dittoed manuscript of 396 pages New Jersey: Department of Statistics, Princeton University.

  43. Verbeke, G., and Molenberghs, G. (2000), Linear Mixed Models for Longitudinal Data, Berlin: Springer.

    MATH  Google Scholar 

  44. Williams, D. A. (1971), “A Test for Differences Between Treatment Means When Several Dose Levels Are Compared with a Zero Dose Control,” Biometrics, 27, 103–117.

    Article  Google Scholar 

  45. Xie, C. C. (2012), “Weighted Multiple Testing Correction for Correlated Tests,” Statistics in Medicine, 31 (4), 341–352.

    MathSciNet  Article  Google Scholar 

  46. Xu, H. Y., Nuamah, I., Liu, J. Y., Lim, P., and Sampson, A. (2009), “A Dunnett-Bonferroni-Based Parallel Gatekeeping Procedure for Dose-Response Clinical Trials with Multiple Endpoints,” Pharmaceutical Statistics, 8 (4), 301–316.

    Google Scholar 

  47. Zimmermann, N. S., Gerendás, J. and Krumbein, A. (2007), “Identification of Desulphoglucosinolates in Brassicaceae by LC/MS/MS: Comparison of ESI and Atmospheric Pressure Chemical Ionisation-MS,” Molecular Nutrition and Food Research, 51, 1537–1546.

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Mario Hasler.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Hasler, M., Böhlendorf, K. Multiple Comparisons for Multiple Endpoints in Agricultural Experiments. JABES 18, 578–593 (2013). https://doi.org/10.1007/s13253-013-0149-7

Download citation

Key Words

  • Correlated endpoints
  • Multiple contrast tests
  • Multiplicity adjustment
  • Simultaneous confidence intervals