Cautionary note on calculating standardized effect size (SES) in randomization test
- 8 Downloads
In community ecology, randomization tests with problem specific test statistics (e.g., nestedness, functional diversity, etc.) are often applied. Researchers in such studies may want not only to detect the significant departure from randomness, but also to measure the effect size (i.e., the magnitude of this departure). Measuring the effect size is necessary, for instance, when the roles of different assembly forces (e.g., environmental filtering, competition) are compared among sites. The standard method is to calculate standardized effect size (SES), i.e., to compute the departure from the mean of random communities divided by their standard deviations. Standardized effect size is a useful measure if the test statistic (e.g., nestedness index, phylogenetic or functional diversity) in the random communities follows a symmetric distribution. In this paper, I would like to call attention to the fact that SES may give us misleading information if the distribution is asymmetric (skewed). For symmetric distribution median and mean values are equal (i.e., SES = 0 indicates p = 0.5). However, this condition does not hold for skewed distributions. For symmetric distributions departure from the mean shows the extremity of the value, regardless of the sign of departure, while in asymmetric distributions the same deviation can be highly probable and extremely improbable, depending on its sign. To avoid these problems, I recommend checking symmetry of null-distribution before calculating the SES value. If the distribution is skewed, I recommend either log-transformation of the test statistic, or using probit-transformed p-value as effect size measure.
KeywordsFunctional diversity Randomization test Standardized effect size Statistics
Cumulative Distribution Function
Leaf Dry Matter Content
Rao’s Quadratic entropy
Standardized Effect Size
Specific Leaf Area
Thanks to M. Scotti, B. Lhotsky and two anonymous reviewers for their comments that helped to improve the manuscript.
- Gotelli, N.J. and G.R. Graves. 1996. Null Models in Ecology. Smithsonian Institution Press, Washington, D.C.Google Scholar
- Gotelli, N.J. and D.J. McCabe. 2002. Species co-occurrence: a me-ta-analysis of J. M. Diamond’s assembly rules model. Ecology 83:2091–2096.Google Scholar
- Götzenberger, L., F. de Bello, K.A. Bråthen, J. Davison, A. Dubuis, A. Guisan, J. Lepš, R. Lindborg, M. Moora, M. Pärtel, L. Pellissier, J. Pottier, P. Vittoz, K. Zobel and M. Zobel. 2012. Ecological assembly rules in plant communities—approaches, patterns and prospects. Biol. Rev. 87:111–127.CrossRefGoogle Scholar
- Kraft, N.J.B., L.S. Comita, J.M. Chase, N.J. Sanders, N.G. Swenson, T.O. Crist, J.C. Stegen, M. Vellend, B. Boyle, M.J. Anderson, H.V. Cornell, K.F. Davies, A.L. Freestone, B.D. Inouye, S.P. Harrison and J.A. Myers. 2011. Disentangling the drivers of β diversity along latitudinal and elevational gradients. Science 333:1755–1758.CrossRefGoogle Scholar
- Manly, B.F.J. 1997. Randomization, Bootstrap and Monte Carlo Methods in Biology. Second Edition. Chapman & Hall, London.Google Scholar
- R Core Team. 2013. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL: www.R-project.org
- Rothstein, H., M. Borenstein, L.V. Hedges and J.P.T. Higgins. 2013. Introduction to Meta-analysis. Wiley, Hoboken, N.J.Google Scholar
- Signorell, A. 2015. DescTools: Tools for descriptive statistics.Google Scholar
Open Access. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited, you give a link to the Creative Commons License, and indicate if changes were made.