Agricultural experiments often have a completely randomized design, and multiple, correlated variables are measured. This paper addresses an appropriate statistical evaluation. A multivariate t-distribution is used for the calculation of multiplicity-adjusted p-values and simultaneous confidence intervals. The number of the multiple variables as well as their correlations are taken into account this way. We consider ratios of means instead of differences, and comparisons versus the overall mean instead of all-pair comparisons. A data set from a greenhouse experiment with glucosinolates of several cultivars of Chinese cabbage (Brassica rapa subsp. pekinensis) is used as an example. Related code based on the R-package SimComp is presented. This package allows a wide application in many agricultural experiments with a similar design.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Bandyopadhyay, S., Ganguli, B., and Chatterjee, A. (2011), “A Review of Multivariate Longitudinal Data Analysis,” Statistical Methods in Medical Research, 20 (4), 299–330.
Bretz, F. (2006), “An Extension of the Williams Trend Test to General Unbalanced Linear Models,” Computational Statistics & Data Analysis, 50 (7), 1735–1748.
Bretz, F., Genz, A., and Hothorn, L. A. (2001), “On the Numerical Availability of Multiple Comparison Procedures,” Biometrical Journal, 43, 645–656.
Buonaccorsi, J. P., and Iyer, H. K. (1984), “A Comparison of Confidence Regions and Designs in Estimation of a Ratio,” Communications in Statistics. Theory and Methods, 13, 723–741.
Dilba, G., Bretz, F., Guiard, V., and Hothorn, L. A. (2004), “Simultaneous Confidence Intervals for Ratios with Applications to the Comparison of Several Treatments with a Control,” Methods of Information in Medicine, 43 (5), 465–469.
Djira, G. D., and Hothorn, L. A. (2009), “Detecting Relative Changes in Multiple Comparisons with an Overall Mean,” Journal of Quality Technology, 41, 60–65.
Dunnett, C. W. (1955), “A Multiple Comparison Procedure for Comparing Several Treatments with a Control,” Journal of the American Statistical Association, 50 (272), 1096–1121.
Faes, C., Molenberghs, G., Aerts, M., Verbeke, G., and Kenward, M. G. (2009), “The Effective Sample Size and an Alternative Small-Sample Degrees-of-Freedom Method,” American Statistician, 63 (4), 389–399.
Fieller, E. C. (1954), “Some Problems in Interval Estimation,” Journal of the Royal Statistical Society, Series B, 16, 175–185.
Fieuws, S., Verbeke, G., and Mollenberghs, G. (2007), “Random-Effects Models for Multivariate Repeated Measures,” Statistical Methods in Medical Research, 16 (5), 387–397.
Frömke, C., and Bretz, F. (2004), “Simultaneous Tests and Confidence Intervals for the Evaluation of Agricultural Field Trials,” Agronomy Journal, 96 (5), 1323–1330.
Genz, A., and Bretz, F. (2002), “Methods for the Computation of Multivariate T-Probabilities,” Journal of Computational and Graphical Statistics, 11, 950–971.
Genz, A., Bretz, F., Miwa, T., Mi, X., Leisch, F., Scheipl, F., and Hothorn, T. (2012), mvtnorm: Multivariate Normal and t Distributions. R package version 0.9-9994, available at http://CRAN.R-project.org/package=mvtnorm.
Gerendas, J., Breuning, S., Stahl, T., Mersch-Sundermann, V., and Mühling, K. H. (2008), “Isothiocyanate Concentration in Kohlrabi (Brassica Oleracea L. Var. Gongylodes) Plants as Influenced by Sulfur and Nitrogen Supply,” Journal of Agricultural and Food Chemistry, 56 (18), 8334–8342.
Guilbaud, O. (2011), “Note on Simultaneous Inferences About Non-Inferiority and Superiority for a Primary and a Secondary Endpoint,” Biometrical Journal, 53, 6 (SI), 927–937.
Hasler, M. (2012), SimComp: Simultaneous Comparisons for Multiple Endpoints. R package version 1.7.0, available at http://CRAN.R-project.org/package=SimComp.
Hasler, M., and Hothorn, L. A. (2008), “Multiple Contrast Tests in the Presence of Heteroscedasticity,” Biometrical Journal, 50 (5), 793–800.
— (2011), “A Dunnett-Type Procedure for Multiple Endpoints,” The International Journal of Biostatistics, 7 (1), 3.
— (2012), “A Multivariate Williams-Type Trend Procedure,” Statistics in Biopharmaceutical Research, 4, 57–65.
Hochberg, Y. (1988), “A Sharper Bonferroni Procedure for Multiple Tests of Significance,” Biometrika, 75 (4), 800–802.
Holm, S. (1979), “A Simple Sequentially Rejective Multiple Test Procedure,” Scandinavian Journal of Statistics, 6, 65–70.
Hommel, G. (1988), “A Stagewise Rejective Multiple Test Procedure Based on a Modified Bonferroni Test,” Biometrika, 75 (2), 383–386.
Hothorn, T., Bretz, F., and Genz, A. (2001), “On Multivariate t and Gauss Probabilities in R,” R News, 1 (2), 27–29.
Hsu, J. (1996), “Multiple Comparisons: Theory and Methods,” 1 edn., London: Chapman & Hall/CRC.
Huang, P., Tilley, B. C., Woolson, R. F., and Lipsitz, S. (2005), “Adjusting O’Brien’s Test to Control Type i Error for the Generalized Nonparametric Behrens-Fisher Problem,” Biometrics, 61 (2), 532–539.
ICH E9 Expert Working Group (1999), “ICH Harmonised Tripartite Guideline: Statistical Principles for Clinical Trials,” Statistics in Medicine, 18 (15), 1905–1942.
Konietschke, F., Hothorn, L. A., and Brunner, E. (2012), “Rank-Based Multiple Test Procedures and Simultaneous Confidence Intervals,” Electronic Journal of Statistics, 6, 738–759.
Laird, N. M., and Ware, J. H. (1982), “Random-Effects Models for Longitudinal Data,” Biometrics, 38 (4), 963–974.
Läuter, J., Glimm, E., and Kropf, S. (1996), “New Multivariate Tests for Data with an Inherent Structure,” Biometrical Journal, 38 (1), 5–23.
— (1998), “Multivariate Tests Based on Left-Spherically Distributed Linear Scores,” The Annals of Statistics, 26 (5), 1972–1988.
Liu, Y., Hsu, J., and Ruberg, S. (2007), “Partition Testing in Dose-Response Studies with Multiple Endpoints,” Pharmaceutical Statistics, 6 (3), 181–192.
Liu, W., Ah-Kine, P., Bretz, F., and Hayter, A. J. (2013), “Exact Simultaneous Confidence Intervals for a Finite Set of Contrasts of Three, Four or Five Generally Correlated Normal Means,” Computational Statistics & Data Analysis, 57, 141–148.
Mithen, R. F., Dekker, M., Verkerk, R., Rabot, S., and Johnson, I. T. (2000), “The Nutritional Significance, Biosynthesis and Bioavailability of Glucosinolates in Human Foods,” Journal of the Science of Food and Agriculture, 80 (7), 967–984.
Nelson, P. R. (1989), “Multiple Comparisons of Means Using Simultaneous Confidence Intervals,” Journal of Quality Technology, 21 (4), 232–241.
Neuhäuser, M. (2006), “How to Deal with Multiple Endpoints in Clinical Trials,” Fundamental & Clinical Pharmacology, 20 (6), 515–523.
R Core Team (2012), R: a Language and Environment for Statistical Computing, Vienna: R Foundation for Statistical Computing. ISBN 3-900051-07-0. URL: http://www.R-project.org/.
Rangkadilok, N., Nicolas, M. E., Bennett, R. N., Eagling, D. R., Premier, R. R., and Taylor, P. W. J. (2004), “The Effect of Sulfur Fertilizer on Glucoraphanin Levels in Broccoli (B. Oleracea L. Var. Italica) at Different Growth Stages,” Journal of Agricultural and Food Chemistry, 52 (9), 2632–2639.
Schaarschmidt, F., and Vaas, L. (2009), “Analysis of Trials with Complex Treatment Structure Using Multiple Contrast Tests,” HortScience, 44 (1), 188–195.
Schonhof, I., Blankenburg, D., Müller, S., and Krumbein, A. (2007), “Sulfur and Nitrogen Supply Influence Growth, Product Appearance, and Glucosinolate Concentration of Broccoli,” Journal of Plant Nutrition and Soil Science, 170 (1), 65–72.
Strassburger, K., and Bretz, F. (2008), “Compatible Simultaneous Lower Confidence Bounds for the Holm Procedure and Other Bonferroni-Based Closed Tests,” Statistics in Medicine, 27 (24), 4914–4927.
The MathWorks Inc. (2010), MATLAB, Natick, Massachusetts. Version 7.10.0 (R2010a).
Tukey, J. W. (1953), The Problem of Multiple Comparisons Dittoed manuscript of 396 pages New Jersey: Department of Statistics, Princeton University.
Verbeke, G., and Molenberghs, G. (2000), Linear Mixed Models for Longitudinal Data, Berlin: Springer.
Williams, D. A. (1971), “A Test for Differences Between Treatment Means When Several Dose Levels Are Compared with a Zero Dose Control,” Biometrics, 27, 103–117.
Xie, C. C. (2012), “Weighted Multiple Testing Correction for Correlated Tests,” Statistics in Medicine, 31 (4), 341–352.
Xu, H. Y., Nuamah, I., Liu, J. Y., Lim, P., and Sampson, A. (2009), “A Dunnett-Bonferroni-Based Parallel Gatekeeping Procedure for Dose-Response Clinical Trials with Multiple Endpoints,” Pharmaceutical Statistics, 8 (4), 301–316.
Zimmermann, N. S., Gerendás, J. and Krumbein, A. (2007), “Identification of Desulphoglucosinolates in Brassicaceae by LC/MS/MS: Comparison of ESI and Atmospheric Pressure Chemical Ionisation-MS,” Molecular Nutrition and Food Research, 51, 1537–1546.
About this article
Cite this article
Hasler, M., Böhlendorf, K. Multiple Comparisons for Multiple Endpoints in Agricultural Experiments. JABES 18, 578–593 (2013). https://doi.org/10.1007/s13253-013-0149-7
- Correlated endpoints
- Multiple contrast tests
- Multiplicity adjustment
- Simultaneous confidence intervals