Advances in Health Sciences Education

, Volume 9, Issue 3, pp 241–249 | Cite as

A Visitor's Guide to Effect Sizes – Statistical Significance Versus Practical (Clinical) Importance of Research Findings

Article

Abstract

Effect Sizes (ES) are an increasingly important index used toquantify the degree of practical significanceof study results. This paper gives anintroduction to the computation andinterpretation of effect sizes from theperspective of the consumer of the researchliterature. The key points made are:1. ES is a useful indicator of the practical(clinical) importance of research resultsthat can be operationally defined frombeing ``negligible'' to ``moderate'', to``important''.2. The ES has two advantages overstatistical significance testing: (a) itis independent of the size of the sample;(b) it is a scale-free index. Therefore,ES can be uniformly interpreted indifferent studies regardless of the samplesize and the original scales of thevariables.3. Calculations of the ES are illustrated byusing examples of comparisons between twomeans, correlation coefficients,chi-square tests and two proportions,along with appropriate formulas.4. Operational definitions for the ESs aregiven, along with numerical examples forthe purpose of illustration.

clinical significance effect size medical education research practical importance 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. American Psychological Association (2001). Publication Manual of the American Psychological Association (5th edition). Washington, DC: American Psychological Association.Google Scholar
  2. Cohen, J. (1987). Statistical Power Analysis for Behavioral Sciences. Hillsdale, NJ: Erlbaum.Google Scholar
  3. Owen, S.B. (1962). Handbook of Statistical Tables. Reading, MA: Addison-Wesley.Google Scholar
  4. Dunlap, W.P., Cortina, J.M., Vaslow, J.B. & Burke, M.J. (1996). Meta-analysis of experiments with matched groups or repeated measure designs. Psychological Methods 1: 170–177.Google Scholar
  5. Friedman, H. (1968). Magnitude of experimental effect and a table for its rapid estimation.Psychological Bulletin 70: 245–253.Google Scholar
  6. Gibbons, R.D., Hedeker, D.R. & Davis, J.M. (1993). Estimation of effect size from a series of experiments involving paired comparisons. Journal of Educational Statistics 18: 271–279.Google Scholar
  7. Glass, G.V., McGaw, B. & Smith, M.L. (1981). Meta-Analysis in Social Research. Beverly Hills, CA.Google Scholar
  8. Hedges, L.V. (1981). Distribution theory for Glass's estimator of effect size and related measures.Journal of Educational Statistics 6: 107–128.Google Scholar
  9. Hedges, L.V. (1982). Estimation of effect size from a series of independent experiments. Psychological Bulletin 92: 490–499.Google Scholar
  10. Hedges, L.V. & Olkin, I. (1985). Statistical Methods for Meta-Analysis. New York: Academic Press.Google Scholar
  11. Jacobson, N.S. & Truax, P. (1991). Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology 59: 12–19.Google Scholar
  12. Kirk, R. (1996). Practical significance: a concept whose time has come. Educational and Psychological Measurement 56: 746–759.Google Scholar
  13. Morris, S.B. (2002). Distribution of the standardized mean change effect size for meta-analysis on repeated measures. British Journal of Mathematical and Statistical Psychology 53: 17–29.Google Scholar
  14. Morris, S.B. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods 7: 105–125.Google Scholar
  15. Rosenthal, R. & Rosnow, R.L. (1991). Essentials of Behavioral Research: Methods and Data Analysis. New York: McGraw Hill.Google Scholar
  16. Rosnow, R.L. & Rosenthal, R. (1996). Computing contrasts, effect sizes, and counternulls on other people's published data: general procedures for research consumers. Psychological Methods 1: 331–340.Google Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  1. 1.Center for Research in Medical Education and Health CareUSA
  2. 2.Department of Psychiatry and Human BehaviorJefferson Medical College of Thomas Jefferson UniversityUSA
  3. 3.Jefferson Medical CollegePhiladelphiaUSA

Personalised recommendations