Behavioral Ecology and Sociobiology

, Volume 65, Issue 1, pp 57–68 | Cite as

Issues in information theory-based statistical inference—a commentary from a frequentist’s perspective

Review

Abstract

After several decades during which applied statistical inference in research on animal behaviour and behavioural ecology has been heavily dominated by null hypothesis significance testing (NHST), a new approach based on information theoretic (IT) criteria has recently become increasingly popular, and occasionally, it has been considered to be generally superior to conventional NHST. In this commentary, I discuss some limitations the IT-based method may have under certain circumstances. In addition, I reviewed some recent articles published in the fields of animal behaviour and behavioural ecology and point to some common failures, misunderstandings and issues frequently appearing in the practical application of IT-based methods. Based on this, I give some hints about how to avoid common pitfalls in the application of IT-based inference, when to choose one or the other approach and discuss under which circumstances a mixing of the two approaches might be appropriate.

Keywords

Akaike’s information criterion Null hypothesis significance testing Data dredging 

References

  1. Aiken LS, West SG (1991) Multiple regression: testing and interpreting interactions. Sage, Newbury ParkGoogle Scholar
  2. American Psychological Association (1994) Publication manual of the American Psychological Association, 4th edn. APA, WashingtonGoogle Scholar
  3. Anderson DR, Burnham KP, Thompson WL (2000) Null hypothesis testing: problems, prevalence, and an alternative. J Wildl Manage 64:912–923CrossRefGoogle Scholar
  4. Anderson DR, Link WA, Johnson DH, Burnham KP (2001) Suggestions for presenting the results of data analyses. J Wildl Manage 65:373–378CrossRefGoogle Scholar
  5. Austin PC, Tu JV (2004) Automated variable selection methods for logistic regression produced unstable models for predicting acute myocardial infarction mortality. J Clin Epidemiol 57:1138–1146PubMedCrossRefGoogle Scholar
  6. Burnham KP, Anderson DR (2002) Model selection and multimodel inference, 2nd edn. Springer, BerlinGoogle Scholar
  7. Burnham KP, Anderson DR, Huyvaert KP (2010) AICc model selection in Ecological and behavioral science: some background, observations, and comparisons. Behav Ecol Sociobiol. doi:10.1007/s00265-010-1029-6
  8. Chatfield C (1995) Model uncertainty, data mining and statistical inference. J Roy Stat Soc A Sta 158:419–466CrossRefGoogle Scholar
  9. Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. Lawrence Erlbaum Associates, New YorkGoogle Scholar
  10. Cohen J (1994) The earth is round (p < .05). Am Psychol 49:997–1003CrossRefGoogle Scholar
  11. Cohen J, Cohen P (1983) Applied multiple regression/correlation analysis for the behavioral sciences, 2nd edn. Lawrence Erlbaum Associates, MahwahGoogle Scholar
  12. Derksen S, Keselman HJ (1992) Backward, forward and stepwise automated subset selection algorithms: frequency of obtaining authentic and noise variables. Br J Math Stat Psychol 45:265–282Google Scholar
  13. Dobson AJ (2002) An introduction to generalized linear models. Chapman & Hall, Boca RatonGoogle Scholar
  14. Dochtermann NA, Jenkins SH (2010) Developing multiple hypotheses in behavioral ecology. Behav Ecol Sociobiol. doi:10.1007/s00265-010-1039-4
  15. Field A (2005) Discovering statistics using SPSS. Sage, LondonGoogle Scholar
  16. Forstmeier W, Schielzeth H (2010) Cryptic multiple hypotheses testing in linear models: overestimated effect sizes and the winner’s curse. Behav Ecol Sociobiol. doi:10.1007/s00265-010-1038-5
  17. Freckleton RP (2010) Dealing with collinearity in behavioural and ecological data: model averaging and the problems of measurement error. Behav Ecol Sociobiol. doi:10.1007/s00265-010-1045-6
  18. Freedman DA (1983) A note on screening regression equations. Am Stat 37:152–155CrossRefGoogle Scholar
  19. Garamszegi LZ (2010) Information-theoretic approaches to statistical analysis in behavioural ecology: an introduction. Behav Ecol Sociobiol. doi:10.1007/s00265-010-1028-7
  20. Garamszegi LZ, Calhim S, Dochtermann N, Hegyi G, Hurd PL, Jørgensen C, Kutsukake N, Lajeunesse MJ, Pollard KA, Schielzeth H, Symonds MRE, Nakagawa S (2009) Changing philosophies and tools for statistical inferences in behavioral ecology. Behav Ecol 20:1363–1375CrossRefGoogle Scholar
  21. Guthery FS, Brennan LA, Peterson MJ, Lusk JJ (2005) Information theory in wildlife science: critique and viewpoint. J Wildl Manage 69:457–465CrossRefGoogle Scholar
  22. Harrell FE Jr (2001) Regression modeling strategies. Springer, New YorkGoogle Scholar
  23. Hector A, von Felten S, Schmid B (2010) Analysis of variance with unbalanced data: an update for ecology & evolution. J Anim Ecol 79:308–316PubMedCrossRefGoogle Scholar
  24. Hegyi G, Garamszegi LZ (2010) Using information theory as a substitute for stepwise regression in ecology and behavior. Behav Ecol Sociobiol. doi:10.1007/s00265-010-1036-7
  25. Hurlbert SH, Lombardi CM (2009) Final collapse of the Neyman–Pearson decision theoretic framework and rise of the neoFisherian. Ann Zool Fenn 46:311–349Google Scholar
  26. Ioannidis JPA (2005) Why most published research findings are false. PLoS Med 2:696–701Google Scholar
  27. James FC, McCulloch CE (1990) Multivariate analysis in ecology and systematics: panacea or Pandora’s box? Annu Rev Ecol Evol Syst 21:129–166Google Scholar
  28. Johnson DH (1999) The insignificance of statistical significance testing. J Wildl Manage 63:763–772CrossRefGoogle Scholar
  29. Johnson DH (2002) The role of hypothesis testing in wildlife science. J Wildl Manage 66:272–276CrossRefGoogle Scholar
  30. Johnson BJ, Omland KS (2004) Model selection in ecology and evolution. Trends Ecol Evol 19:101–108PubMedCrossRefGoogle Scholar
  31. Lovell MC (1983) Data mining. Rev Econ Stat 65:1–12CrossRefGoogle Scholar
  32. Lukacs PM, Thompson WL, Kendall WL, Gould WR, Doherty PF Jr, Burnham KP, Anderson DR (2007) Concerns regarding a call for pluralism of information theory and hypothesis testing. J Appl Ecol 44:456–460CrossRefGoogle Scholar
  33. McCullagh P, Nelder JA (2008) Generalized linear models. Chapman and Hall, LondonGoogle Scholar
  34. Møller AP, Jennions MD (2002) How much variance can be explained by ecologists and evolutionary biologists? Oecologia 132:492–500CrossRefGoogle Scholar
  35. Mundry R, Nunn CL (2009) Stepwise model fitting and statistical inference: turning noise into signal pollution. Am Nat 173:119–123PubMedCrossRefGoogle Scholar
  36. Nakagawa S, Cuthill IC (2007) Effect size, confidence interval and statistical significance: a practical guide for biologists. Biol Rev 82:591–605PubMedCrossRefGoogle Scholar
  37. Nickerson RS (2000) Null hypothesis significance testing: a review of an old and continuing controversy. Psychol Methods 5:241–301PubMedCrossRefGoogle Scholar
  38. Quinn GP, Keough MJ (2002) Experimental designs and data analysis for biologists. Cambridge University Press, CambridgeGoogle Scholar
  39. R Development Core Team (2009) R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, AustriaGoogle Scholar
  40. Richards SA, Whittingham MJ, Stephens PA (2010) Model selection and model averaging in behavioural ecology: the utility of the IT-AIC framework. Behav Ecol Sociobiol. doi:10.1007/s00265-010-1035-8
  41. Royall R (1997) Statistical evidence, a likelihood paradigm. Chapman & Hall, LondonGoogle Scholar
  42. Sakamoto Y, Akaike H (1978) Analysis of cross classified data by AIC. Ann Inst Stat Math 30:185–197CrossRefGoogle Scholar
  43. Schielzeth H (2010) Simple means to improve the interpretability of regression coefficients. Methods Ecol Evol 1:103–113CrossRefGoogle Scholar
  44. Siegel S, Castellan NJ (1988) Nonparametric statistics for the behavioral sciences, 2nd edn. McGraw-Hill, New YorkGoogle Scholar
  45. Sleep DJH, Drever MC, Nudds TD (2007) Statistical versus biological hypothesis testing: response to Steidl. J Wildl Manage 71:2120–2121CrossRefGoogle Scholar
  46. Smith GD, Ebrahim S (2002) Data dredging, bias, or confounding. Brit Med J 325:1437–1438PubMedCrossRefGoogle Scholar
  47. Sokal RR, Rohlf FJ (1995) Biometry—the principles and practice of statistics in biological research, 3rd edn. Freeman, New YorkGoogle Scholar
  48. Steidl RJ (2006) Model selection, hypothesis testing, and risks of condemning analytical tools. J Wildl Manage 70:1497–1498CrossRefGoogle Scholar
  49. Stephens PA, Buskirk SW, Hayward GD, del Rio CM (2005) Information theory and hypothesis testing: a call for pluralism. J Appl Ecol 42:4–12CrossRefGoogle Scholar
  50. Stephens PA, Buskirk SW, del Rio CM (2007) Inference in ecology and evolution. Trends Ecol Evol 22:192–197PubMedCrossRefGoogle Scholar
  51. Stoehr AM (1999) Are significance thresholds appropriate for the study of animal behaviour? Anim Behav 57:F22–F25PubMedCrossRefGoogle Scholar
  52. Symonds MRE, Moussalli A (2010) A brief guide to model selection, multimodel inference and model averaging in behavioural ecology using Akaike’s Information Criterion. Behav Ecol Sociobiol. doi:10.1007/s00265-010-1037-6
  53. Tabachnick BG, Fidell LS (2001) Using multivariate statistics, 4th edn. Allyn & Bacon, BostonGoogle Scholar
  54. Vul E, Harris C, Winkielman P, Pashler H (2010) Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition. Perspect Psychol Sci 4:274–290CrossRefGoogle Scholar
  55. Whittingham MJ, Stephens PA, Bradbury RB, Freckleton RP (2006) Why do we still use stepwise modelling in ecology and behaviour? J Anim Ecol 75:1182–1189PubMedCrossRefGoogle Scholar
  56. Young SS, Bang H, Oktay K (2009) Cereal-induced gender selection? Most likely a multiple testing false positive. Proc R Soc Lond, Ser B 276:1211–1212CrossRefGoogle Scholar
  57. Zar JH (1999) Biostatistical analysis, 4th edn. Prentice Hall, New JerseyGoogle Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  1. 1.Max Planck Institute for Evolutionary AnthropologyLeipzigGermany

Personalised recommendations