, Volume 11, Issue 4, pp 851–860 | Cite as

Experiment design beyond gut feeling: statistical tests and power to detect differential metabolites in mass spectrometry data

  • Diana TrutschelEmail author
  • Stephan Schmidt
  • Ivo Grosse
  • Steffen Neumann
Original Article


Univariate hypotheses tests such as Student’s t test or variance analysis (ANOVA) can help to answer a variety of questions in metabolomics data analysis. The statistical power of these tests depends on the setup of the experiment, the experimental design and the analytical variance of the actual observations. In this paper, we demonstrate how a well-designed pilot study prior to an experiment with the aim to find differences between e.g. several genotypes, can help to determine the variance at multiple levels ranging from biological variance, sample preparation to instrumental variances. Next, we illustrate how these variances can be used to obtain several parameters (e.g. minimum statistically significant effect, number of required replicates and error probabilities) which influence the design of the actual study. In particular, we are going to sketch how technical replicates can improve the performance of a test, when they are correctly used in the statistical analysis, e.g. with a hierarchical model. Finally, we demonstrate the process of evaluating the trade-off between different experimental designs with different replication strategies. The choice of an experimental design beyond the gut feeling can be influenced by factors such as costs, sample availability and the accuracy of of the tests. We use metabolite profiles of the model plant Arabidopsis thaliana measured on an UPLC-ESI/QqTOF-MS as real-world dataset, but the approach is equally applicable to other sample types and measurement methods like NMR based metabolomics.


Metabolomics Statistics Variances Hierarchical experiment design 


Conflict of interest

The authors declare that they have no conflict of interest.

Compliance with ethical requirements

This article does not contain any studies with human or animal subjects.

Supplementary material

11306_2014_742_MOESM1_ESM.pdf (317 kb)
Supplementary material 1 (PDF 318 KB)
11306_2014_742_MOESM2_ESM.r (12 kb)
Supplementary material 1 (R 13 KB)
11306_2014_742_MOESM3_ESM.pdf (250 kb)
Supplementary material 1 (PDF 251 KB)
11306_2014_742_MOESM4_ESM.rnw (17 kb)
Supplementary material 1 (RNW 18 KB)


  1. Ahrens, Heinz. (1967). Varianzanalyse. Berlin: Akademieverlag WTB.Google Scholar
  2. Baldi, P., & Long, A. D. (2001). A Bayesian framework for the analysis of microarray expression data: Regularized t test and statistical inferences of gene changes. Bioinformatics, 17(6), 509–519.PubMedCrossRefGoogle Scholar
  3. Böttcher, C., von Roepenack-Lahaye, E., & Scheel, D. (2011) Genetics and genomics of the Brassicaceae, crops and models ( Vol XII). In: Resources for metabolomics (p. 677). New York: SpringerGoogle Scholar
  4. Böttcher, C., Westphal, L., Schmotz, C., Prade, E., Scheel, D., & Glawischnig, E. (2009). The multifunctional enzyme CYP71B15 (PHYTOALEXIN DEFICIENT3) converts cysteine-indole-3-acetonitrile to camalexin in the indole-3-acetonitrile metabolic network of Arabidopsis thaliana. The Plant Cell Online, 21(6), 1830–1845.CrossRefGoogle Scholar
  5. Broadhurst, D. I., & Kell, D. B. (2006). Statistical strategies for avoiding fals discoveries in metabolomics and related experiments. Metabolomics, 2(2):171–196.Google Scholar
  6. Danielsson, A. P. H., Moritz, T., Mulder, H., & Spegel, P. (2012). Development of a gas chromatography/mass spectrometry based metabolomics protocol by means of statistical experimental design. Metabolomics, 8, 50–63.CrossRefGoogle Scholar
  7. Davis, C. (2002). Statistical methods for the analysis of repeated measurements. New York: Springer.Google Scholar
  8. Donner, A. (1996). Statistical considerations in the design and analysis of community intervention trials. Journal of Clinical Epidemiology, 49(4), 435–439.PubMedCrossRefGoogle Scholar
  9. Dreyhaupt, Jens., Sufeida, Sabrina., & Muche, Rainer. Power- und Fallzahlabschätzungen für hierarchische und longitudinale Studien. In 17. Konferenz der SAS-Anwender in Forschung und Entwicklung. KSFE e.V., 03 (2013).Google Scholar
  10. Dunn, W. B. (2008). Current trends and future requirements for the mass spectrometric investigation of microbial, mammalian and plant metabolomes. Physical Biology, 5(1), 011001. (24pp).PubMedCrossRefGoogle Scholar
  11. Dunn, W., Erban, A., Weber, R., Creek, D., Brown, M., Breitling, R., et al. (2013). Mass appeal: Metabolite identification in mass spectrometry-focused untargeted metabolomics. Metabolomics, 9, 44–66. doi: 10.1007/s11306-012-0434-4.CrossRefGoogle Scholar
  12. Eliasson, M., Rännar, S., Madsen, R., Donten, M. A., Marsden-Edwards, E., Moritz, T., et al. (2012). Strategy for optimizing LC-MS data processing in metabolomics: A design of experiments approach. Analytical Chemistry, 84(15), 6869–6876.PubMedCrossRefGoogle Scholar
  13. Goodacre, R., Broadhurst, D., Smilde, A. K., Kristal, B. S., Baker, J. D., Beger, R., et al. (2007). Proposed minimum reporting standards for data analysis in metabolomics. Metabolomics, 3(3), 231–241.CrossRefGoogle Scholar
  14. Haug, K., Salek, R. M., Conesa, P., Hastings, J., de Matos, P., Rijnbeek, M., et al. (2013). MetaboLights—an open-access general-purpose repository for metabolomics studies and associated meta-data. Nucleic Acids Research, 41(Database issue), D781–D786.PubMedCentralPubMedCrossRefGoogle Scholar
  15. Hendriks, M. M. W. B., van Eeuwijk, F. A., Jellema, R. H., Westerhuis, J. A., Reijmers, T. H., Hoefsloot, H. C. J., et al. (2011). Data-processing strategies for metabolomics studies. Trends in Analytical Chemistry, 30(10), 1685–1698.CrossRefGoogle Scholar
  16. Holmes, T. H. (2004). Ten categories of statistical errors: A guide for research in endocrinology and metabolism. American Journal of Physiology–Endocrinology and Metabolism, 286(4), E495–E501.PubMedCrossRefGoogle Scholar
  17. Horgan, G. W. (2007). Sample size and replication in 2D gel electrophoresis studies. Journal of Proteome Research, 6(7), 2884–2887.PubMedCrossRefGoogle Scholar
  18. Johnson, H. E., Lloyd, A. J., Mur, L. A., Smith, A. R., & Causton, D. R. (2007). The application of MANOVA to analyse Arabidopsis thaliana metabolomic data from factorially designed experiments. Metabolomics, 3, 517–530.CrossRefGoogle Scholar
  19. Karp, N. A., Spencer, M., Lindsay, H., O’Dell, K., & Lilley, K. S. (2005). Impact of replicate types on proteomic expression analysis. Journal of Proteome Research, 4(5), 1867–1871.PubMedCrossRefGoogle Scholar
  20. Lönnstedt, I., & Speed, T. (2001). Replicated microarray data. Statistica Sinica, 12, 31–46.Google Scholar
  21. Pavlidis, P., Li, Q., & Stafford, N. W. (2003). The effect of replication on gene expression microarray experiments. Bioinformatics, 19(13), 1620–1627.PubMedCrossRefGoogle Scholar
  22. Pinheiro, J. C., & Bates, D. (2014). Mixed-effects models in S and S-PLUS. New York: Springer.Google Scholar
  23. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks: SAGE.Google Scholar
  24. Saccenti, E., Hoefsloot, H. C., Smilde, A. K., Westerhuis, J. A., & Hendriks, M. M. (2013). Reflections on univariate and multivariate analysis of metabolomics data. Metabolomics, 1–14.Google Scholar
  25. Sampson, J. N., Boca, S. M., Shu, X. O., Stolzenberg-Solomon, R. Z., Matthews, C. E., Hsing, A. W., et al. (2013). Metabolomics in epidemiology: Sources of variability in metabolite measurements and implications. Cancer Epidemiology Biomarkers & Prevention, 22(4), 631–640.CrossRefGoogle Scholar
  26. Smith, C. A., Want, E. J., O’Maille, G., Abagyan, R., & Siuzdak, G. (2006). XCMS: Processing mass spectrometry data for metabolite profiling using nonlinear peak alignment, matching and identification. Analytical Chemistry, 78(3), 779–787.PubMedCrossRefGoogle Scholar
  27. Snijders, T. A. B. (2001). Sampling, Chapter 11. In A. Leyland & H. Goldstein (Eds.), Multilevel modelling of health statistics (pp. 159–174). Longford: Wiley.Google Scholar
  28. Snijders, Tom A. B., & Snijders, T. A. (2005). Power and sample size in multilevel linear models. Encyclopedia of Statistics in Behavioral Science, 3, 1570–1573.Google Scholar
  29. Student, (1908). The probable error of a mean. Biometrika, 6, 1–25.CrossRefGoogle Scholar
  30. Tutz, G., Fahrmeir, L., & Hamerle, A. (1996). Multivariate statistische verfahren. Berlin: Walter de Gryuter.Google Scholar
  31. Vinaixa, M., Samino, S., Saez, I., Duran, J., Guinovart, J. J., & Yanes, O. (2012). A guideline to univariate statistical analysis for LC/MS-based untargeted metabolomics-derived data. Metabolites, 2(4), 775–795.PubMedCentralPubMedCrossRefGoogle Scholar
  32. von Roepenack-Lahaye, E., Degenkolb, T., Zerjeski, M., Franz, M., Roth, U., Wessjohann, L., et al. (2004). Profiling of Arabidopsis secondary metabolites by capillary liquid chromatography coupled to electrospray ionization quadrupole time-of-flight mass spectrometry. Plant Physiology, 134(2), 548–559.PubMedCentralPubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Diana Trutschel
    • 1
    Email author
  • Stephan Schmidt
    • 1
  • Ivo Grosse
    • 2
    • 3
  • Steffen Neumann
    • 1
  1. 1.Department of Stress and Developmental BiologyLeibniz Institute of Plant BiochemistryHalleGermany
  2. 2.Institute of Computer ScienceMartin-Luther-University Halle-WittenbergHalleGermany
  3. 3.German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-LeipzigLeipzigGermany

Personalised recommendations