Abstract
This paper discusses the value of quasi-experiments with small numbers of cases - small-n case studies. It argues that these evaluations of crime prevention have value. The first part of the paper shows that small-n evaluations are practical methods for producing information and that discarding this information reduces our ability to learn what works in crime prevention. The solution is to examine small-n evaluations in systematic reviews. In the second part this paper shows that more rigorous evaluations can sacrifice generalizability to gain internal validity and that this, too, reduces our ability to learn. The solution is to create a mixed portfolio of intrusive, less-intrusive, and non-intrusive evaluations. In the third part this paper shows why inference processes that make use of theory should replace naïve inductionism. One implication of this is that ignoring theory also discards evidence. Another implication is that taking theory into account when we review program effectiveness allows us to better use small-n evaluations when we generalize. In the final part, this paper gives three recommendations for making use of small-n evaluations: adopt case study procedures developed in political science, improve the value of process evaluations, and improve crime theories.
This is a preview of subscription content, access via your institution.
References
Brady, H. E. (2004). Data-set observations versus causal-process observations: The 2000 U.S. presidential elections. In H. E. Brady, & D. Collier, D. (Eds.), Rethinking social inquiry (pp. 267–271). Oxford: Rowman and Littlefield.
Brady, H. E. & Collier, D. (Eds.). (2004). Rethinking social inquiry. Oxford: Rowman and Littlefield.
Braga, A. A., Weisburd, D. L., Waring, E. J., Mazerolle, L. G., Spelman, W. & Gajewski, F. (1999). Problem-oriented policing in violent crime places: A randomized controlled experiment. Criminology 37(3), 541–580.
Campbell, D. T. (1988a). Administrative experimentation, institutional records, and nonreactive measures. In D. T. Campbell & E. S. Overman (Eds.), Methodology and epistemology for social science: Selected papers (pp. 243-260). Chicago: University of Chicago Press.
Campbell, D. T. (1988b). Can we be scientific in applied social science? In D. T. Campbell & E. S. Overman (Eds.), Methodology and epistemology for social science: Selected papers (pp. 315–333). Chicago: University of Chicago Press.
Campbell, D. T. (1988c). Qualitative knowing in action research. In D. T. Campbell & E. S. Overman (Eds.), Methodology and epistemology for social science: Selected papers (pp. 360–376). Chicago: University of Chicago Press.
Clarke, R. V. & Cornish, D. B. (1972). The controlled trial in institutional research: Paradigm or pitfall for penal evaluators? London: Her Majesty's Stationery Office.
Dugan, L., LaFree, G. & Piquero, A. R. (2005). Testing a rational choice model of airline hijackings. Criminology 43(4), 1031–1065.
Eck, J. E. (2005). Evaluation for lesson learning. In N. Tilley (Ed.), Handbook of crime prevention and community safety (pp. 699–733). Cullompton, Devon: Willan.
Eck, J. E. & Wartell, J. (1998). Improving the management of rental properties with drug problems: A randomized experiment. In L. Mazerolle & J. Roehl (Eds.), Civil remedies and crime prevention (Vol. 9). Monsey, NY: Criminal Justice Press.
Farrington, D., Gottfredson, D. C., Sherman, L. W. & Welsh, B. (2002). The Maryland scientific methods scale. In: L. W. Sherman, D. Farrington, B. Welsh & D. L. MacKenzie (Eds.), Evidence-based crime prevention (pp. 13–21). New York: Routledge.
Felson, M., Belange, M. E., Bichler, G. M., Bruzinski, C. D., Campbell, G. S., Fried, C. L., et al. (1996). Redesigning hell: Preventing crime and disorder at the Port Authority bus terminal. In R. V. Clarke (Ed.), Preventing mass transit crime (Vol. 6) (pp. 5–92). Monsey, NY: Criminal Justice Press.
Garner, J. H., Fagan, J. & Maxwell, C. D. (1995). Published findings from the spouse assault replication program: A critical review. Journal of Quantitative Criminology 11(1), 3–28.
George, A. L. & Bennett, A. (2005). Case studies and theory development in the social sciences. Cambridge, MA: MIT Press.
Goertz, G. & Starr, H. (Eds.). (2003). Necessary conditions: Theory, methodology, and applications. Lanham, MD: Rowman & Littlefield.
Hacking, I. (2001). An introduction to probability and inductive Logic. New York: Cambridge University Press.
Holland, J. H., Holyoak, K. J., Nisbett, R. E. & Thagard, P. R. (1986). Induction: Processes of inference, learning, and discovery. Cambridge, MA: MIT Press.
King, G., Keohane, R. O. & Verba, S. (1994). Designing social inquiry. Princeton, NJ: Princeton University Press.
La Vigne, N. G. (1996). Safe transport: Security by design on the Washington Metro. In R. V. Clarke (Ed.), Preventing mass transit crime (Vol. 6) (pp. 163–198). Monsey, NY: Criminal Justice Press.
Lipton, P. (2004). Inference to the best explanation (2nd ed.). London: Routledge.
Lum, C. & Yang, S.-M. (2005). Why do evaluation researchers in crime and justice choose non-experimental methods? Journal of Experimental Criminology 1(2), 191–213.
Madensen, T., Fisher, B. S. & Eck, J. E. (2004). Cinco de Mayo student disturbances: University of Cincinnati follow-up survey. Cincinnati, OH: Center for Criminal Justice Research, University of Cincinnati www.uc.edu/criminaljustice/ProjectReports/FOLLOWUP_CINCO_REPORT.pdf.
Madensen, T. & Morgan, D. (2005). Evaluation of traffic barricade impact on crime in Pendelton: Cincinnati, Ohio. Cincinnati, OH: Criminal Justice Research Center, University of Cincinnati www.uc.edu/criminaljustice/ProjectReports/BARRICADE_REPORT.pdf.
Maxwell, C. D., Garner, J. H. & Fagan, J. A. (2002). The preventive effects of arrest on intimate partner violence: Research, policy and theory. Criminology and Public Policy 2(1), 51–80.
National Research Council. (2005). Improving evaluation of anticrime programs. Washington, DC: National Academies Press.
Pawson, R. & Tilley, N. (2000). Realistic evaluation. New York: Sage.
Popper, K. R. (1992). Conjectures and refutations: The growth of scientific knowledge. New York: Routledge.
Shadish, W. R., Cook, T. D. & Campbell, D. T. (2002). Experimentation and quasi-experimental designs for general causal inference. New York: Houghton Mifflin.
Sherman, L. W. & Weisburd, D. A. (1995). General deterrent effects of police patrol in crime “hot spots”: A randomized, controlled trial. Justice Quarterly 12(4), 625–648.
Webb, E. J., Campbell, D. T., Schwartz, R. D. & Sechrest, L. (1966). Unobtrusive measures: Nonreactive research in the social sciences. Chicago, IL: Rand McNally.
Weisburd, D., Lum, C. M. & Petrosino, A. (2001). Does research design affect study outcomes in criminal justice? The Annals of the American Academy of Political and Social Science 578, 50–70, (November).
Weiss, C. (1997). Theory-based evaluation: Past, present, and future. In D. J. Rog & D. Fournier (Eds.), Progress and future directions in evaluation: Perspectives on theory, practice and methods (pp. 41–56). San Francisco: Jossey-Bass.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Eck, J.E. When is a bologna sandwich better than sex? A defense of small-n case study evaluations. J Exp Criminol 2, 345–362 (2006). https://doi.org/10.1007/s11292-006-9014-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11292-006-9014-9
Key words
- case studies
- evaluation
- external validity
- induction
- problem-oriented policing
- quasi-experiments
- randomized experiments
- situational crime prevention
- systematic reviews