Skip to main content

Advertisement

SpringerLink
Log in

Bayesian analysis and the accumulation of evidence in crime and justice intervention studies

  • Published:
Journal of Experimental Criminology Aims and scope Submit manuscript

Abstract

In recent years a great deal of attention has turned to the need for policy-relevant research in criminology. Methodologically, attention has been trained on the use of randomized experimental designs and cumulative systematic reviews of evidence to accomplish this goal. Our work here reviews and demonstrates the utility of the Bayesian analytic framework, in the context of crime prevention and justice treatment studies, as a means of furthering the goals of research synthesis and creation of policy-relevant scientific statements. Evidence from various fields is used as a foundation for the discussion, and an empirical example illustrates how this approach might be useful in practical criminological research. It is concluded that Bayesian analysis offers a useful complement to existing approaches and warrants further inclusion in the ongoing discussion about how best to assess program effectiveness, synthesize evidence, and report findings from crime and justice evaluations in a way that is relevant to policy makers and practitioners.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. See, for example, recent discussions by Sherman (2003) and Turner et al. (2003) that have emphasized the importance of the Cochrane Collaboration as a model for the Campbell Collaboration.

  2. Or, at the very least, a justification for suggesting an uninformed prior state of belief.

  3. Although we chose this progression of studies for the illustrative analysis, there are other possible sequences that may be utilized with these data. For example, in the incremental approach to such analyses, chronological ordering would likely be used. Where all studies are available simultaneously, analysts would have to be clear about the rationale for the sequencing of the studies and potential sensitivity of results to that ordering. One might also examine a sequenced analysis in relation to a pooled approach, as is done here.

  4. Odds ratios reflect the exponentiation of the log odds (e b).

  5. This approach was exhibited in the Berk et al. (1992b) study of results from multiple experiments of police response to domestic violence incidents. Essentially, they display the shift in observed effects as a result of an increase or decrease in prior information, which, in their case, entailed information from the prior studies.

  6. Although it should be pointed out that the interpretation of results would be different from a frequentist analysis.

  7. This lack of exposure is supported in a recent examination of statistical training in doctoral programs in criminology and criminal justice, which affirms the hegemony of NHST approaches and found little mention of other analytic paradigms (Sullivan and Maxfield 2003).

  8. See Berk (2004, 2005), Kline (2004), and Maltz (1994, 2006) for discussions that highlight some ways in which present methods could use more thoughtful consideration.

  9. See Spiegelhalter et al. (2004) for a discussion on how to combine and weight data from various sources in Bayesian analysis.

References

  • Abelson, R. P. (1995). Statistics as principled argument. Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Barnard, G. A. (1958). Studies in the history of probability and statistics, IX: Thomas Bayes’ essay toward solving a problem in the doctrine of chances. Biometrika, 45, 293–295.

    Google Scholar 

  • Berk, R. A. (2004). Regression analysis: a constructive critique. Newbury Park, CA: Sage.

    Google Scholar 

  • Berk, R. A. (2005). Randomized experiments as the bronze standard. Journal of Experimental Criminology, 1, 417–433.

    Article  Google Scholar 

  • Berk, R. A., Campbell, A., Klap, R., & Western, B. (1992a). The deterrent effect of arrests in incidents of domestic violence: a Bayesian analysis of four field experiments. American Sociological Review, 57, 698–708.

    Article  Google Scholar 

  • Berk, R. A., Campbell, A., Klap, R., & Western, B. (1992b). Bayesian analysis of the Colorado Springs spouse abuse experiments. Journal of Criminal Law and Criminology, 83, 170–200.

    Article  Google Scholar 

  • Berk, R. A., Western, B., & Weiss, R. E. (1995). Statistical inference of apparent populations. Sociological Methodology, 25, 421–458.

    Article  Google Scholar 

  • Boruch, R. (2007). The null hypothesis is not called that for nothing: statistical tests in randomized trials. Journal of Experimental Criminology, 3, 1–20.

    Article  Google Scholar 

  • Boruch, R., Snyder, B., & DeMoya, D. (2000). The importance of randomized trials. Crime & Delinquency, 46, 156–180.

    Article  Google Scholar 

  • Bushway, S. D., Sweeten, G., & Wilson, D. B. (2006). Size matters: Standard errors in the application of null hypothesis significance testing in criminology and criminal justice. Journal of Experimental Criminology, 2, 1–22.

    Article  Google Scholar 

  • Campbell, D. T. (1969). Reforms as experiments. American Psychologist, 24, 409–429.

    Article  Google Scholar 

  • Chalmers, I. (2003). Trying to do more good than harm in policy and practice: the role of rigorous, transparent, up-to-date evaluations. The Annals of the American Academy of Political and Social Science, 589, 22–40.

    Article  Google Scholar 

  • Chalmers, I., Hedges, L. V., & Cooper, H. (2002). A brief history of research synthesis. Evaluation & The Health Professions, 25, 12–37.

    Article  Google Scholar 

  • Clear, T. R., & Frost, N. A. (2001). Criminology and public policy: a new journal of the American Society of Criminology. Criminology & Public Policy, 1, 1–3.

    Article  Google Scholar 

  • Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997–1003.

    Article  Google Scholar 

  • Dunson, D. B. (2001). Practical advantages of Bayesian analysis of epidemiologic data. American Journal of Epidemiology, 153, 1222–1226.

    Article  Google Scholar 

  • Etzioni, R. D., & Kadane, J. B. (1995). Bayesian statistical methods in public health and medicine. Annual Review of Public Health, 16, 23–41.

    Article  Google Scholar 

  • Farrington, D. P., & Weisburd, D. L. (2007). The Campbell Collaboration crime and justice group. The Criminologist, 32(1), 3–5. 1.

    Google Scholar 

  • Farrington, D. P., & Welsh, B. C. (2005). Randomized experiments in criminology: what have we learned in the last two decades? Journal of Experimental Criminology, 1, 9–38.

    Article  Google Scholar 

  • Freedman, D. (1995). Some issues in the foundation of statistics. Foundations of Science, 1, 19–83.

    Google Scholar 

  • Freedman, L. (1996). Bayesian statistical methods: a natural way to assess clinical evidence. British Medical Journal, 313, 569–570.

    Google Scholar 

  • Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003). Bayesian data analysis (2nd ed.). London, UK: Chapman & Hall.

    Google Scholar 

  • Gill, J. (2002). Bayesian methods: a social and behavioral sciences approach. London, UK: Chapman & Hall.

    Google Scholar 

  • Goodman, S. N. (1999a). Towards evidence based medical statistics, 1: the p value fallacy. Annals of Internal Medicine, 130, 995–1004.

    Google Scholar 

  • Goodman, S. N. (1999b). Towards evidence based medical statistics, 2: the Bayes factor. Annals of Internal Medicine, 130, 1005–1013.

    Google Scholar 

  • Harrell, A. V. (2006). Towards systematic knowledge building: an anti-crime research and development continuum. Journal of Experimental Criminology, 2, 339–344.

    Article  Google Scholar 

  • Hays, W. L., & Winkler, R. L. (1970). Statistics: Probability, inference, and decision, Vol. 1. New York, NY: Holt, Rinehart, and Winston.

    Google Scholar 

  • Howard, G. S., Maxwell, S. E., & Fleming, K. J. (2000). The proof of pudding: An illustration of the relative strengths of null hypothesis, meta-analysis, and Bayesian analysis. Psychological Methods, 5, 315–332.

    Article  Google Scholar 

  • Iversen, G. R. (1984). Bayesian statistical inference. Beverly Hills, CA: Sage.

    Google Scholar 

  • Jefferys, W. H., & Berger, J. O. (1992). Ockham’s razor and Bayesian analysis. American Scientist, 80, 64–72.

    Google Scholar 

  • Kadane, J. B. (1995). Prime time for Bayes. Controlled Clinical Trials, 16, 313–318.

    Article  Google Scholar 

  • Kline, R. B. (2004). Beyond significance testing: Reforming data analysis methods in behavioral research. Washington, D.C.: American Psychological Association.

    Google Scholar 

  • Laplace, P. S. (1986). Memoir on the probability of the causes of events (reprint from 1774). Statistical Science, 1, 364–378.

    Article  Google Scholar 

  • Lee, P. (1997). Bayesian statistics: An introduction (2nd ed.). London: Edward Arnold Press.

    Google Scholar 

  • Lilford, R. J., & Braunholz, D. (1996). The statistical basis of public policy: a paradigm shift is overdue. British Medical Journal, 313, 603–608.

    Google Scholar 

  • Lilford, R. J., & Braunholz, D. (2000). The Cochrane lecture: who’s afraid of Thomas Bayes? Journal of Epidemiology and Community Health, 54, 731–739.

    Article  Google Scholar 

  • Lipsey, M. W., Adams, J., Gottfredson, D. C., Pepper, J., & Weisburd, D. (2005). Improving evaluation of anticrime programs. Washington, D.C.: National Academies Press.

    Google Scholar 

  • Lipsey, M., Petrie, C., Weisburd, D., & Gottfredson, D. (2006). Improving evaluation of anti-crime programs: Summary of a National Research Council report. Journal of Experimental Criminology, 2, 271–307.

    Article  Google Scholar 

  • Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.

    Google Scholar 

  • Maltz, M. D. (1994). Deviating from the mean: the declining significance of significance. Journal of Research in Crime and Delinquency, 31, 434–463.

    Article  Google Scholar 

  • Maltz, M. D. (2006). Some p-baked thoughts (p > 0.5) on experiments and statistical significance. Journal of Experimental Criminology, 2, 211–226.

    Article  Google Scholar 

  • Marden, J. I. (2000). Hypothesis testing: from p values to Bayes factors. Journal of the American Statistical Association, 95, 1316–1320.

    Article  Google Scholar 

  • Nagin, D. S. (2005). Group-based modeling of development. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Petersilia, J., & Turner, S. (1993). Intensive probation and parole. In M. Tonry (Ed.), Crime and justice: A review of research (Vol. 17, pp. 281–335). Chicago: University of Chicago.

    Google Scholar 

  • Petrie, C., & Weisburd, D. (2006). Introduction to the special issue on the evaluation of anti-crime programs. Journal of Experimental Criminology, 2, 265–270.

    Article  Google Scholar 

  • Pilz, J. (1991). Bayesian estimation and experimental design in linear regression models. New York: John Wiley & Sons.

    Google Scholar 

  • Pollard, W. E. (1986). Bayesian statistics for evaluation research: An introduction. Beverly Hills, CA: Sage.

    Google Scholar 

  • Raudenbush, S. W., & Bryk, A. (2002). Hierarchical linear models (2nd ed.). Thousand Oaks, CA: Sage.

    Google Scholar 

  • Rosenthal, R. (1990). Replication in behavioral research. Journal of Social Behavior and Personality, 5, 1–30.

    Google Scholar 

  • Sheingold, S. H. (2001). Can Bayesian methods make data and analyses more relevant to decision makers? International Journal of Technology Assessment in Health Care, 17, 114–122.

    Article  Google Scholar 

  • Sherman, L. W. (2003). Misleading evidence and evidence-led policy: making social science more experimental. The Annals of the American Academy of Political and Social Science, 589, 6–19.

    Article  Google Scholar 

  • Spiegelhalter, D. J., Abrams, K. R., & Myles, J. P. (2004). Bayesian approaches to clinical trials and health care evaluation. New York, NY: Wiley.

    Google Scholar 

  • Spiegelhalter, D. J., Freedman, L. S., & Parmar, M. K. B. (1994). Bayesian approaches to randomized trials. Journal of the Royal Statistical Society (A), 157, 357–416.

    Article  Google Scholar 

  • Sullivan, C. J., & Maxfield, M. G. (2003). Examining paradigmatic development in criminology and criminal justice: a content analysis of research methods syllabi in doctoral programs. Journal of Criminal Justice Education, 14, 269–285.

    Article  Google Scholar 

  • Turner, H., Boruch, R., Petrosino, A., Lavenberg, J., DeMoya, D., & Rothstein, H. (2003). Populating an international web-based randomized trials register in the social, behavioral, criminological, and education sciences. The Annals of the American Academy of Political and Social Science, 589, 203–223.

    Article  Google Scholar 

  • van der Wilt, G. J., Rovers, M., Straatman, H., Van den Broek, P., & Zielhuis, G. (2004). Policy relevance of Bayesian statistics overestimated? International Journal of Technology Assessment in Health Care, 20, 488–492.

    Article  Google Scholar 

  • Weisburd, D., Lum, C. M., & Yang, S. M. (2003a). When can we conclude that treatments or programs “don’t work”? The Annals of the American Academy of Political and Social Science, 587, 31–48.

    Article  Google Scholar 

  • Weisburd, D., Mazerolle, L., & Petrosino, A. (2007). The academy of experimental criminology: advancing randomized trials in crime and justice. The Criminologist, 32(3), 3–7. 1.

    Google Scholar 

  • Weisburd, D., Petrosino, A., & Lum, C. (2003b). Preface. The Annals of the American Academy of Political and Social Science, 587, 6–15.

    Article  Google Scholar 

  • Weiss, R., Berk, R., Li, W., & Farrell-Ross, M. (1999). Death penalty charging in Los Angeles County: an illustrative data analysis using skeptical priors. Sociological Methods & Research, 28, 91–115.

    Article  Google Scholar 

  • Western, B. (1999a). Guest editor’s introduction to the special issue on Bayesian methods in the social sciences. Sociological Methods & Research, 28, 3–6.

    Article  Google Scholar 

  • Western, B. (1999b). Bayesian analysis for sociologists: an introduction. Sociological Methods & Research, 28, 7–34.

    Article  Google Scholar 

  • Winkler, R. L. (2001). Why Bayesian analysis hasn’t caught on in healthcare decision making. International Journal of Technology Assessment in Health Care, 17, 56–66.

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank Michael Maxfield, Yancy Edwards, Alex Piquero, and Jean McGloin as well as the editor and anonymous reviewers for their helpful comments on earlier drafts of this paper.

Author information

Authors and Affiliations

  1. Department of Criminology, University of South Florida, 4202 E Fowler Ave, SOC 326, Tampa, FL, 33620, USA

    Christopher J. Sullivan & Thomas Mieczkowski

Authors
  1. Christopher J. Sullivan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Mieczkowski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christopher J. Sullivan.

Additional information

Data utilized in this paper were drawn from Petersilia, Joan, Susan Turner, and Elizabeth Piper Deschenes. Intensive supervision for high-risk offenders in 14 sites in the United States, 1987–1990 (Computer file). ICPSR version. Santa Monica, CA, USA: The RAND Corporation (producer), 1994. Ann Arbor, MI, USA: Inter-university Consortium for Political and Social Research (distributor), 1999.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sullivan, C.J., Mieczkowski, T. Bayesian analysis and the accumulation of evidence in crime and justice intervention studies. J Exp Criminol 4, 381–402 (2008). https://doi.org/10.1007/s11292-008-9062-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11292-008-9062-4

Keywords

Search

Navigation