Issues in information theorybased statistical inference—a commentary from a frequentist’s perspective
 Roger Mundry
 … show all 1 hide
Rent the article at a discount
Rent now* Final gross prices may vary according to local VAT.
Get AccessAbstract
After several decades during which applied statistical inference in research on animal behaviour and behavioural ecology has been heavily dominated by null hypothesis significance testing (NHST), a new approach based on information theoretic (IT) criteria has recently become increasingly popular, and occasionally, it has been considered to be generally superior to conventional NHST. In this commentary, I discuss some limitations the ITbased method may have under certain circumstances. In addition, I reviewed some recent articles published in the fields of animal behaviour and behavioural ecology and point to some common failures, misunderstandings and issues frequently appearing in the practical application of ITbased methods. Based on this, I give some hints about how to avoid common pitfalls in the application of ITbased inference, when to choose one or the other approach and discuss under which circumstances a mixing of the two approaches might be appropriate.
 Aiken, LS, West, SG (1991) Multiple regression: testing and interpreting interactions. Sage, Newbury Park
 Publication manual of the American Psychological Association. APA, Washington
 Anderson, DR, Burnham, KP, Thompson, WL (2000) Null hypothesis testing: problems, prevalence, and an alternative. J Wildl Manage 64: pp. 912923 CrossRef
 Anderson, DR, Link, WA, Johnson, DH, Burnham, KP (2001) Suggestions for presenting the results of data analyses. J Wildl Manage 65: pp. 373378 CrossRef
 Austin, PC, Tu, JV (2004) Automated variable selection methods for logistic regression produced unstable models for predicting acute myocardial infarction mortality. J Clin Epidemiol 57: pp. 11381146 CrossRef
 Burnham, KP, Anderson, DR (2002) Model selection and multimodel inference. Springer, Berlin
 Burnham KP, Anderson DR, Huyvaert KP (2010) AICc model selection in Ecological and behavioral science: some background, observations, and comparisons. Behav Ecol Sociobiol. doi:10.1007/s0026501010296
 Chatfield, C (1995) Model uncertainty, data mining and statistical inference. J Roy Stat Soc A Sta 158: pp. 419466 CrossRef
 Cohen, J (1988) Statistical power analysis for the behavioral sciences. Lawrence Erlbaum Associates, New York
 Cohen, J (1994) The earth is round (p < .05). Am Psychol 49: pp. 9971003 CrossRef
 Cohen, J, Cohen, P (1983) Applied multiple regression/correlation analysis for the behavioral sciences. Lawrence Erlbaum Associates, Mahwah
 Derksen, S, Keselman, HJ (1992) Backward, forward and stepwise automated subset selection algorithms: frequency of obtaining authentic and noise variables. Br J Math Stat Psychol 45: pp. 265282
 Dobson, AJ (2002) An introduction to generalized linear models. Chapman & Hall, Boca Raton
 Dochtermann NA, Jenkins SH (2010) Developing multiple hypotheses in behavioral ecology. Behav Ecol Sociobiol. doi:10.1007/s0026501010394
 Field, A (2005) Discovering statistics using SPSS. Sage, London
 Forstmeier W, Schielzeth H (2010) Cryptic multiple hypotheses testing in linear models: overestimated effect sizes and the winner’s curse. Behav Ecol Sociobiol. doi:10.1007/s0026501010385
 Freckleton RP (2010) Dealing with collinearity in behavioural and ecological data: model averaging and the problems of measurement error. Behav Ecol Sociobiol. doi:10.1007/s0026501010456
 Freedman, DA (1983) A note on screening regression equations. Am Stat 37: pp. 152155 CrossRef
 Garamszegi LZ (2010) Informationtheoretic approaches to statistical analysis in behavioural ecology: an introduction. Behav Ecol Sociobiol. doi:10.1007/s0026501010287
 Garamszegi, LZ, Calhim, S, Dochtermann, N, Hegyi, G, Hurd, PL, Jørgensen, C, Kutsukake, N, Lajeunesse, MJ, Pollard, KA, Schielzeth, H, Symonds, MRE, Nakagawa, S (2009) Changing philosophies and tools for statistical inferences in behavioral ecology. Behav Ecol 20: pp. 13631375 CrossRef
 Guthery, FS, Brennan, LA, Peterson, MJ, Lusk, JJ (2005) Information theory in wildlife science: critique and viewpoint. J Wildl Manage 69: pp. 457465 CrossRef
 Harrell, FE (2001) Regression modeling strategies. Springer, New York
 Hector, A, Felten, S, Schmid, B (2010) Analysis of variance with unbalanced data: an update for ecology & evolution. J Anim Ecol 79: pp. 308316 CrossRef
 Hegyi G, Garamszegi LZ (2010) Using information theory as a substitute for stepwise regression in ecology and behavior. Behav Ecol Sociobiol. doi:10.1007/s0026501010367
 Hurlbert, SH, Lombardi, CM (2009) Final collapse of the Neyman–Pearson decision theoretic framework and rise of the neoFisherian. Ann Zool Fenn 46: pp. 311349
 Ioannidis, JPA (2005) Why most published research findings are false. PLoS Med 2: pp. 696701
 James, FC, McCulloch, CE (1990) Multivariate analysis in ecology and systematics: panacea or Pandora’s box?. Annu Rev Ecol Evol Syst 21: pp. 129166
 Johnson, DH (1999) The insignificance of statistical significance testing. J Wildl Manage 63: pp. 763772 CrossRef
 Johnson, DH (2002) The role of hypothesis testing in wildlife science. J Wildl Manage 66: pp. 272276 CrossRef
 Johnson, BJ, Omland, KS (2004) Model selection in ecology and evolution. Trends Ecol Evol 19: pp. 101108 CrossRef
 Lovell, MC (1983) Data mining. Rev Econ Stat 65: pp. 112 CrossRef
 Lukacs, PM, Thompson, WL, Kendall, WL, Gould, WR, Doherty, PF, Burnham, KP, Anderson, DR (2007) Concerns regarding a call for pluralism of information theory and hypothesis testing. J Appl Ecol 44: pp. 456460 CrossRef
 McCullagh, P, Nelder, JA (2008) Generalized linear models. Chapman and Hall, London
 Møller, AP, Jennions, MD (2002) How much variance can be explained by ecologists and evolutionary biologists?. Oecologia 132: pp. 492500 CrossRef
 Mundry, R, Nunn, CL (2009) Stepwise model fitting and statistical inference: turning noise into signal pollution. Am Nat 173: pp. 119123 CrossRef
 Nakagawa, S, Cuthill, IC (2007) Effect size, confidence interval and statistical significance: a practical guide for biologists. Biol Rev 82: pp. 591605 CrossRef
 Nickerson, RS (2000) Null hypothesis significance testing: a review of an old and continuing controversy. Psychol Methods 5: pp. 241301 CrossRef
 Quinn, GP, Keough, MJ (2002) Experimental designs and data analysis for biologists. Cambridge University Press, Cambridge
 R Development Core Team (2009) R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria
 Richards SA, Whittingham MJ, Stephens PA (2010) Model selection and model averaging in behavioural ecology: the utility of the ITAIC framework. Behav Ecol Sociobiol. doi:10.1007/s0026501010358
 Royall, R (1997) Statistical evidence, a likelihood paradigm. Chapman & Hall, London
 Sakamoto, Y, Akaike, H (1978) Analysis of cross classified data by AIC. Ann Inst Stat Math 30: pp. 185197 CrossRef
 Schielzeth, H (2010) Simple means to improve the interpretability of regression coefficients. Methods Ecol Evol 1: pp. 103113 CrossRef
 Siegel, S, Castellan, NJ (1988) Nonparametric statistics for the behavioral sciences. McGrawHill, New York
 Sleep, DJH, Drever, MC, Nudds, TD (2007) Statistical versus biological hypothesis testing: response to Steidl. J Wildl Manage 71: pp. 21202121 CrossRef
 Smith, GD, Ebrahim, S (2002) Data dredging, bias, or confounding. Brit Med J 325: pp. 14371438 CrossRef
 Sokal, RR, Rohlf, FJ (1995) Biometry—the principles and practice of statistics in biological research. Freeman, New York
 Steidl, RJ (2006) Model selection, hypothesis testing, and risks of condemning analytical tools. J Wildl Manage 70: pp. 14971498 CrossRef
 Stephens, PA, Buskirk, SW, Hayward, GD, Rio, CM (2005) Information theory and hypothesis testing: a call for pluralism. J Appl Ecol 42: pp. 412 CrossRef
 Stephens, PA, Buskirk, SW, Rio, CM (2007) Inference in ecology and evolution. Trends Ecol Evol 22: pp. 192197 CrossRef
 Stoehr, AM (1999) Are significance thresholds appropriate for the study of animal behaviour?. Anim Behav 57: pp. F22F25 CrossRef
 Symonds MRE, Moussalli A (2010) A brief guide to model selection, multimodel inference and model averaging in behavioural ecology using Akaike’s Information Criterion. Behav Ecol Sociobiol. doi:10.1007/s0026501010376
 Tabachnick, BG, Fidell, LS (2001) Using multivariate statistics. Allyn & Bacon, Boston
 Vul, E, Harris, C, Winkielman, P, Pashler, H (2010) Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition. Perspect Psychol Sci 4: pp. 274290 CrossRef
 Whittingham, MJ, Stephens, PA, Bradbury, RB, Freckleton, RP (2006) Why do we still use stepwise modelling in ecology and behaviour?. J Anim Ecol 75: pp. 11821189 CrossRef
 Young, SS, Bang, H, Oktay, K (2009) Cerealinduced gender selection? Most likely a multiple testing false positive. Proc R Soc Lond, Ser B 276: pp. 12111212 CrossRef
 Zar, JH (1999) Biostatistical analysis. Prentice Hall, New Jersey
 Title
 Issues in information theorybased statistical inference—a commentary from a frequentist’s perspective
 Journal

Behavioral Ecology and Sociobiology
Volume 65, Issue 1 , pp 5768
 Cover Date
 20110101
 DOI
 10.1007/s002650101040y
 Print ISSN
 03405443
 Online ISSN
 14320762
 Publisher
 SpringerVerlag
 Additional Links
 Topics
 Keywords

 Akaike’s information criterion
 Null hypothesis significance testing
 Data dredging
 Industry Sectors
 Authors

 Roger Mundry ^{(1)}
 Author Affiliations

 1. Max Planck Institute for Evolutionary Anthropology, Deutscher Platz 6, 04103, Leipzig, Germany