Skip to main content
SpringerLink
Log in
Book cover

Handbook of Quantitative Criminology pp 181–208Cite as

Meta-analysis

  • Chapter
  • First Online:

Abstract

Meta-analysis is a statistical method of synthesizing quantitative results across studies examining a common research question. This chapter discusses the logic and methods of meta-analysis with specific application to the field of criminology and criminal justice. A key feature of meta-analysis is the effect size, which encodes the study findings on a common index, such as the standardized mean difference, correlation coefficient, or odds-ratio. Analysis of these effect sizes considers not only the central tendency of effects across studies but also the relationship of study features to variability in effects. Both fixed- and random-effects models are discussed, as are the important issue of publication selection bias. Meta-analysis applies social science methodology and statistical methods to the task of taking stock of the evidence in an area, providing a robust foundation for future research and theorizing.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    There is debate within the meta-analytic literature on the relative merits of analyzing the correlation in its raw form or using the Fisher z transformed value (see Field 2001; Hunter and Schmidt 2004). Computer simulations have shown that the raw correlation is slightly downwardly biased but to a lesser degree than the upward bias of the z transformed value. The original purpose, however, of the z transformation was to provide a computable standard error. An alternative approach is to use the raw correlation as the effect-size and approximate the variance as \(v ={ \left (1 - {r}^{2}\right )}^{2}/\left (n - 1\right )\) (e.g., Hunter and Schmidt 2004; Shadish and Haddock 2009).

  2. 2.

    I have served as a consulted in the development of this program.

References

  • Becker BJ (2009) Model-based meta-analysis, 2nd edn, Russell Sage Foundation, New York

    Google Scholar 

  • Berk R (2007) Statistical inference and meta-analysis. J Exp Criminol 3:247–270, doi10.1007/s11292-007-9036-y, URL http://dx.doi.org/10.1007/s11292-007-9036-y

    Google Scholar 

  • Borenstein M (2009) Effect sizes for studies with continuous outcome data, 2nd edn. Russell Sage Foundation, New York

    Google Scholar 

  • Burnett WL (1996) Treating postincarcerated offenders with Moral Reconation TherapyTM: A one-year recidivism study. PhD thesis, University of Phoenix

    Google Scholar 

  • Bushway SD, Sweeten G, Wilson DB (2006) Size matters: standard errors in the application of null hypothesis significance testing in criminology and criminal justice. J Exp Criminol 2:1–22, doi: 10.1007/s11292-005-5129-7, URL http://dx.doi.org/10.1007/s11292-005-5129-7

  • Cheung MWL, Chan W (2005) Meta-analytic structural equation modeling: A two-stage approach. Psychol Methods 10:40–64. doi: 10.1037/1082-989X.10.1.40, URL http://search.ebscohost.com/login.aspx?direct=true/&db=pdh/&AN=met-10-1-40/&site=ehost-live

    Google Scholar 

  • Cohen J, Cohen P (1983) Applied multiple regression/correlation analysis for the behavioral sciences, 2nd edn. L. Erlbaum Associates, Hillsdale, NJ

    Google Scholar 

  • Cooper HM (1998) Synthesizing research: A guide for literature reviews. Sage, Thousand Oaks, CA

    Google Scholar 

  • Cooper HM, DeNeve K, Charlton K (1997) Finding the missing science: The fate of studies submitted for review by a human subjects committee. Psychol Methods 2:447–452. doi: 10.1037/1082-989X.2.4.447, URL http://search.ebscohost.com/login.aspx?direct=true/&db=pdh/&AN=met-2-4-447/&site=ehost-live

  • Cox DR (1970) The analysis of binary data. Methuen, London

    Google Scholar 

  • DerSimonian R, Laird N (1986) Meta-analysis in clinical trials. Control Clin Trials 7:177–88. doi: 3802833, URL http://www.ncbi.nlm.nih.gov/pubmed/3802833

    Google Scholar 

  • Dickersin K (2005) Publication bias: recognizing the problem, understanding its origins, and preventing harm. Wiley, Chichester, pp 11–33

    Google Scholar 

  • Egger M, Smith GD, Schneider M, Minder C (1997) Bias in meta-analysis detected by a simple, graphical test. BMJ 315:629–634, URL http://www.bmj.com/cgi/content/abstract/315/7109/629

  • Field AP (2001) Meta-analysis of correlation coefficients: A monte carlo comparison of fixed- and random-effects methods. Psychol Methods 6:161–180. doi: 10.1037/1082-989X.6.2.161, URL http://search.ebscohost.com/login.aspx?direct=true/&db=pdh/&AN=met-6-2-161/&site=ehost-live

  • Fleiss JL (1994) Measures of effect size for categorical data. Russell Sage Foundation, New York, pp 245–260

    Google Scholar 

  • Fleiss JL, Berlin JA (2009) Measures of effect size for categorical data, 2nd edn. Russell Sage Foundation, New York

    Google Scholar 

  • Friedman L (2000) Estimators of random effects variance components in meta-analysis. J Educ Behav Stat 25:1–12. doi: 10.3102/10769986025001001, URL http://jeb.sagepub.com/cgi/content/abstract/25/1/1

    Google Scholar 

  • Frost JJ, Forrest JD (1995) Understanding the impact of effective teenage pregnancy prevention programs. Fam Plann Perspect 27:188–195, URL http://mutex.gmu.edu:2112/stable/2136274

  • Furlow CF, Beretvas SN (2005) Meta-analytic methods of pooling correlation matrices for structural equation modeling under different patterns of missing data. Psychol Methods 10:227–254, doi: 10.1037/1082-989X.10.2.227, URL http://search.ebscohost.com/login.aspx?direct=true/&db=pdh/&AN=met-10-2-227/&site=ehost-live

  • Gerber AS, Malhotra N (2008) Publication bias in empirical sociological research: Do arbitrary significance levels distort published results? Sociol Methods Res 37:3–30, doi: 10.1177/0049124108318973, URL URL http://smr.sagepub.com/cgi/content/abstract/37/1/3

    Google Scholar 

  • Glass GV (1976) Primary, secondary, and meta-analysis research. Educ Res 5:3–8

    Google Scholar 

  • Gleser LJ, Olkin I (1994) Stochastically dependent effect sizes. Russell Sage Foundation, New York, pp 339–356

    Google Scholar 

  • Hasselblad V, Hedges (1995) Meta-analysis of screening and diagnostic tests. Psychol Bull 117:167–178. doi: 10.1037/0033-2909.117.1.167, URL http://search.ebscohost.com/login.aspx?direct=true/&db=pdh/&AN=bul-117-1-167/&site=ehost-live

  • Hedges LV (1981) Distribution theory for Glass’s estimator of effect size and related estimators. J Educ Behav Stat 6:107–128, doi: 10.3102/10769986006002107, URL http://jeb.sagepub.com/cgi/content/abstract/6/2/107

  • Hedges LV, Olkin I (1985) Statistical methods for meta-analysis. Academic, Orlando, FL

    Google Scholar 

  • Higgins JPT, Thompson SG, Deeks JJ, Altman DG (2003) Measuring inconsistency in meta-analyses. BMJ 327:557–560. doi: 10.1136/bmj.327.7414.557, URL http://www.bmj.com

    Google Scholar 

  • Hunt MM (1997) How science takes stock: The story of meta-analysis. Russell Sage Foundation, New York

    Google Scholar 

  • Hunter JE, Schmidt FL (1990) Methods of meta-analysis: Correcting error and bias in research findings. Sage Publications, Newbury Park

    Google Scholar 

  • Hunter JE, Schmidt FL (2004) Methods of meta-analysis: Correcting error and bias in research findings, 2nd edn. Sage, Thousand Oaks, CA

    Google Scholar 

  • Johnson G, Hunter RM (1995) Evaluation of the specialized drug offender program. In: Ross RR, Ross B (eds) Thinking straight. Cognitive Center, Ottawa, ON, pp 215–234

    Google Scholar 

  • Kalaian HA, Raudenbush SW (1996) A multivariate mixed linear model for meta-analysis. Psychol Methods 1:227–235. doi: 10.1037/1082-989X.1.3.227, URL http://search.ebscohost.com/login.aspx?direct=true/&db=pdh/&AN=met-1-3-227/&site=ehost-live

  • Kuhns JB, Wilson DB, Maguire ER, Ainsworth SA, Clodfelter TA (2008) A meta-analysis of marijuana, cocaine, and opiate toxicology study findings among homicide victims. Addiction 104:1122–1131

    Article  Google Scholar 

  • Lipsey MW (1995) What do we learn from 400 research studies on the effectiveness of treatment with juvenile delinquents? Wiley, New York, pp 63–78

    Google Scholar 

  • Lipsey MW (2007) Unjustified inferences about meta-analysis. J Exp Criminol 3:271–279. doi: 10.1007/s11292-007-9037-x, URL http://dx.doi.org/10.1007/s11292-007-9037-x

    Google Scholar 

  • Lipsey MW, Cullen FT (2007) The effectiveness of correctional rehabilitation: A review of systematic reviews. Annu Rev Law Soc Sci 3:297–320, doi: 10.1146/annurev.lawsocsci.3.081806.112833, URL http://mutex.gmu.edu:2078/doi/full/10.1146/annurev.lawsocsci.%3.081806.112833

    Google Scholar 

  • Lipsey MW, Derzon JH (1998) Predictors of violent or serious delinquency in adolescence and early adulthood: A synthesis of longitudinal research. Sage Publications, Thousand Oaks, CA, pp 86–105

    Google Scholar 

  • Lipsey MW, Wilson DB (1993) The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. Am Psychol 48:1181–1209. doi: 10.1037/0003-066X.48.12.1181, URL http://search.ebscohost.com/login.aspx?direct=true/&db=pdh/&AN=amp-48-12-1181/&site=ehost-live

    Google Scholar 

  • Lipsey MW, Wilson DB (1998) Effective intervention for serious juvenile offenders: A synthesis of research. Sage Publications, Thousand Oaks, CA, pp 313–345

    Google Scholar 

  • Lipsey MW, Wilson DB (2001a) Practical meta-analysis. Applied social research methods series. Sage Publications, Thousand Oaks, CA

    Google Scholar 

  • Lipsey MW, Wilson DB (2001b) The way in which intervention studies have “personality” and why it is important to meta-analysis. Eval Health Prof 24:236–254. doi: 10.1177/016327870102400302, URL http://ehp.sagepub.com/cgi/content/abstract/24/3/236

  • Lipsey MW, Crosse S, Dunkle J, Pollard J, Stobart G (1985) Evaluation: The state of the art and the sorry state of the science. New Dir Program Eval 27:7–28

    Article  Google Scholar 

  • Little GL and Robinson KD (1989) Treating drunk drivers with Moral Reconation therapy: A one-year recidivism report. Psychol Rep 64:960-962

    Google Scholar 

  • Little GL, Robinson KD Burnette KD (1991) Treating drug offenders with Moral Reconation therapy: A three-year recidivism report. Psychol Rep 69:1151–1154

    Google Scholar 

  • Little GL, Robinson KD, Burnette KD (1994) Treating offenders with cognitive-behavioral therapy: 5-year recidivism outcome data on MRT. Cogn Behav Treat Rev 3:1-3

    Google Scholar 

  • Overton RC (1998) A comparison of fixed-effects and mixed (random-effects) models for meta-analysis tests of moderator variable effects. Psychol Methods 3:354–379. doi: 10.1037/1082-989X.3.3.354, URL http://search.ebscohost.com/login.aspx?direct=true/&db=pdh/&AN=met-3-3-354/&site=ehost-live

    Google Scholar 

  • Porporino FJ, Robinson D (1995) An evaluation of the Reasoning and Rehabilitation program with Canadian federal offenders. In: Ross RR, Ross B (eds) Thinking straight. Cognitive Centre, Ottawa, ON, pp 155–191

    Google Scholar 

  • Porporino FJ , Fabiano EA, Robinson D (1991) Focusing on successful reintegration: Cognitive skills training for offenders, r19. Ottawa, Canada: Research and Statistics Branch, The Correctional Service of Canada

    Google Scholar 

  • Pratt TC, Cullen FT (2000) The empirical status of Gottfredson and Hirschi’s General Theory of crime: A meta-analysis. Criminology 38:931–964, doi: 10.1111/j.1745-9125.2000.tb00911.x, URL http://mutex.gmu.edu:2167/doi/abs/10.1111/j.1745-9125.2000.tb%00911.x

  • Raudenbush SW (1994) Random effects models. Russell Sage Foundation, New York, pp 301–322

    Google Scholar 

  • Raudenbush SW (2009) Statistically analyzing effect sizes: Random effects models, 2nd edn. Russell Sage Foundation, New York

    Google Scholar 

  • Robinson D (1995) The impact of cognitive skills training on postrelease recidivism among Canadian federal offenders. Correctional Research and Development, The Correctional Service of Canada, Ottawa, ON

    Google Scholar 

  • Rosenthal R (1991) Meta-analytic procedures for social research, Rev. ed edn. Applied social research methods series. Sage Publications, Newbury Park

    Google Scholar 

  • Ross RR, Fabiano EA, Ewles CD (1988) Reasoning and Rehabilitation. Int J Offender Ther Comp Criminol 32:29-36

    Article  Google Scholar 

  • Schmidt F, Le H, Oh IS (2009) Correcting for the distorting effects of study artifacts in meta-analysis, 2nd edn. Russell Sage Foundation, New York

    Google Scholar 

  • Schulze R (2004) Meta-analysis: A comparison of approaches. Hogrefe & Huber, Toronto

    Google Scholar 

  • Shadish WR (2007) A world without meta-analysis. J Exp Criminol 3:281–291. doi: 10.1007/s11292-007-9034-0, URL http://dx.doi.org/10.1007/s11292-007-9034-0

    Google Scholar 

  • Shadish WR, Haddock CK (2009) Combining estimates of effect size, 2nd edn. Russell Sage Foundation, New York

    Google Scholar 

  • Sánchez-Meca J, Marín-Martínez F, Chacón-Moscoso S (2003) Effect-size indices for dichotomized outcomes in meta-analysis. Psychol Methods 8:448–467, URL jsmeca@um.es

    Google Scholar 

  • Stern JM, Simes RJ (1997) Publication bias: Evidence of delayed publication in a cohort study of clinical research projects. BMJ 315:640–645

    Google Scholar 

  • Sterne JAC, Becker BJ, Egger M (2005) The funnel plot. Wiley, Chichester, pp 75–99

    Google Scholar 

  • Viechtbauer W (2005) Bias and efficiency of meta-analytic variance estimators in the random-effects model. J Educ Behav Stat 30:261–293. doi: 10.3102/10769986030003261, URL http://jeb.sagepub.com/cgi/content/abstract/30/3/261

    Google Scholar 

  • Weisburd D, Lum CM, Yang SM (2003) When can we conclude that treatments or programs “don’t work”? Ann Am Acad Pol Soc Sci 587:31–48, doi: 10.1177/0002716202250782, URL http://ann.sagepub.com/cgi/content/abstract/587/1/31

  • Wilson DB (2009) Missing a critical piece of the pie: Simple document search strategies inadequate for systematic reviews. J Exp Criminol. doi: 10.1007/s11292-009-9085-5

    Google Scholar 

  • Wilson DB, Bouffard LA, Mackenzie DL (2005) A quantitative review of structured, group-oriented, cognitive-behavioral programs for offenders. Crim Justice Behav 32:172–204. doi: 10.1177/0093854804272889, URL http://cjb.sagepub.com/cgi/content/abstract/32/2/172

Download references

Author information

Authors and Affiliations

  1. Administration of Justice Department, George Mason University, Manassas, VA, USA

    David B. Wilson

Authors
  1. David B. Wilson
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. College of Criminology, Florida State University, West Call Street 643, Tallahassee, 32306, U.S.A.

    Alex R. Piquero

  2. Inst. Criminology, Hebrew University of Jerusalem, Jerusalem, 91905, Israel

    David Weisburd

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Wilson, D.B. (2010). Meta-analysis. In: Piquero, A., Weisburd, D. (eds) Handbook of Quantitative Criminology. Springer, New York, NY. https://doi.org/10.1007/978-0-387-77650-7_10

Download citation

Publish with us

Policies and ethics

Search

Navigation