Journal of Quantitative Criminology

, Volume 23, Issue 2, pp 151–178 | Cite as

Is the Magic Still There? The Use of the Heckman Two-Step Correction for Selection Bias in Criminology

  • Shawn Bushway
  • Brian D. Johnson
  • Lee Ann Slocum
Original Paper

Abstract

Issues of selection bias pervade criminological research. Despite their ubiquity, considerable confusion surrounds various approaches for addressing sample selection. The most common approach for dealing with selection bias in criminology remains Heckman’s [(1976) Ann Econ Social Measure 5:475–492] two-step correction. This technique has often been misapplied in criminological research. This paper highlights some common problems with its application, including its use with dichotomous dependent variables, difficulties with calculating the hazard rate, misestimated standard error estimates, and collinearity between the correction term and other regressors in the substantive model of interest. We also discuss the fundamental importance of exclusion restrictions, or theoretically determined variables that affect selection but not the substantive problem of interest. Standard statistical software can readily address some of these common errors, but the real problem with selection bias is substantive, not technical. Any correction for selection bias requires that the researcher understand the source and magnitude of the bias. To illustrate this, we apply a diagnostic technique by Stolzenberg and Relles [(1997) Am Sociol Rev 62:494–507] to help develop intuition about selection bias in the context of criminal sentencing research. Our investigation suggests that while Heckman’s two-step correction can be an appropriate technique for addressing this bias, it is not a magic solution to the problem. Thoughtful consideration is therefore needed before employing this common but overused technique.

Keywords

Sample selection Selection bias Heckman correction Two-step estimator 

Notes

Acknowledgements

The authors would like to thank Justin McCrary, Wayne Osgood, Anne Piehl, Jeff Smith and seminar participants at the Economics and Crime summer workshop for helpful comments. We would also like to thank Gary Sweeten for research assistance and useful discussions. Most of the research for the paper was completed while Bushway was a professor at the University of Maryland. The paper is dedicated to the memory of our friend, mentor, and former colleague Douglas A. Smith. Doug helped us to understand the relevant selection issues, and encouraged us to pursue the paper in its formative stages. He is sorely missed.

References

  1. Ahn H, Powell J (1993) Semiparametric estimation of censored selection models with a nonparametric selection mechanism. J Econom 58:3–29CrossRefGoogle Scholar
  2. Albonetti C (1986) Criminality, prosecutorial screening, and uncertainty: toward a theory of discretionary decision making in felony case processing. Criminology 24:623–644CrossRefGoogle Scholar
  3. Albonetti C (1991) An integration of theories to explain judicial discretion. Soc Probl 38:247–266Google Scholar
  4. Albonetti C (1997) Sentencing under the federal sentencing guidelines: an analysis of the effects of defendant characteristics, guilty pleas, and departures on sentencing outcomes for drug offenses. Law Soc Rev 31:601–634CrossRefGoogle Scholar
  5. Amemiya T (1985) Advanced econometrics. Harvard University Press, Cambridge, MAGoogle Scholar
  6. Angrist J (2001) Estimation of limited dependent variable models with dummy endogenous regressors: simple strategies for empirical practice. J Bus Econ Stat 19:2–16CrossRefGoogle Scholar
  7. Belsley David, Kuh E, Welsch R (1980) Regression diagnostics: identifying influential data and sources of collinearity. John Wiley & Sons, New YorkGoogle Scholar
  8. Berk R (1983) An introduction to sample selection bias in sociological data. Am Soc Rev 48:386–398CrossRefGoogle Scholar
  9. Berk R, Ray S (1982) Selection bias in sociological data. Soc Sci Res 11:352–398CrossRefGoogle Scholar
  10. Blundell R, Powell J (2001) Endogeneity in nonparametric and semiparametric regression models. CEMMAP Working Paper No CWP09/01Google Scholar
  11. Bontrager S, Bales W, Chiricos T (2005) Race, ethnicity, threat and the labeling of convicted felons. Criminology 43:589–622CrossRefGoogle Scholar
  12. Bushway S, Piehl A (2001) Judging judicial discretion: legal factors and racial discrimination in sentencing. Law Soc Rev 35:733–767CrossRefGoogle Scholar
  13. Chay KY, Powell JL (2001) Semiparametric censored regression models. J Econ Persp 15:29–42CrossRefGoogle Scholar
  14. Chesher A, Irish M (1987) Residual analysis in the grouped and censored normal linear model. J Econom 34:33–61CrossRefGoogle Scholar
  15. Chiricos T, Bales W (1991) Unemployment and punishment: an empirical assessment. Criminology 29:701–724CrossRefGoogle Scholar
  16. D’Alessio S, Stolzenberg L (2003) Race and the probability of arrest. Soc Forces 84:1381–1398CrossRefGoogle Scholar
  17. Demuth S (2003) Racial and ethnic differences in pretrial release decisions and outcomes: a comparison of Hispanic, black, and white felony arrestees. Criminology 41:873–908CrossRefGoogle Scholar
  18. Duan N, Manning WG, Morris CN, Newhouse JP (1983) A comparison of alternative models for the demand for medical care. J Econ Bus Stat 1:115–126CrossRefGoogle Scholar
  19. Duan N, Manning WG, Morris CN, Newhouse JP (1984) Choosing between the sample-selection model and the multi-part model. J Bus Econ Stat 2:283–289CrossRefGoogle Scholar
  20. Dubin JA, Rivers D (1990) Selection bias in linear regression, logit and probit models. In: Fox J, Long JS (eds) Modern methods of data analysis. Sage Publications, Thousand Oaks, CA, pp 411–443Google Scholar
  21. Engen RL, Gainey RR (2000) Modeling the effects of legally relevant and extralegal factors under sentencing guidelines: the rules have changed. Criminology 38:1207–1228CrossRefGoogle Scholar
  22. Erez E, Tontodonato P (1990) The effect of victim participation in sentencing on sentencing outcome. Criminology 28:451–474CrossRefGoogle Scholar
  23. Felson RB (1996) Big people hit little people: sex differences in physical power and interpersonal violence. Criminology 34:433–453CrossRefGoogle Scholar
  24. Felson RB, Messner S (1996) To kill or not to kill: lethal outcomes in injurious attacks. Criminology 36:519–545CrossRefGoogle Scholar
  25. Gondolf EW (2000) A 30-month follow-up of court-referred batterers in four cities. Int J Offender Ther Comp Criminol 44:111–128CrossRefGoogle Scholar
  26. Greene WH (1993) Econometric analysis, 2nd edn. Macmillan, New YorkGoogle Scholar
  27. Hagan J, Palloni A (1986) ‘Club Fed’ and the sentencing of white-collar offenders before and after Watergate. Criminology 24:603–622CrossRefGoogle Scholar
  28. Hagan J, Parker P (1985) White-collar crime and punishment: the class structure and legal sanctioning of security violations. Am Sociol Rev 50:302–316CrossRefGoogle Scholar
  29. Halaby C (2004) Panel models in sociological research: theory into practice. Ann Rev Sociol 30Google Scholar
  30. Heckman JJ (1976) The common structure of statistical models of truncation, sample selection and limited dependent variables. Ann Econ Soc Measure 5:475–492Google Scholar
  31. Heckman JJ (1979) Sample selection bias as a specification error. Econometrica 47:153–161CrossRefGoogle Scholar
  32. Heckman JJ, LaLonde R, Smith J. (1999) The economics and econometrics of active labor market programs. In: Ashenfelter O, Card D (eds) Handbook of Labor Economics, vol 3A. North-Holland, Amsterdam, pp 1865–2097Google Scholar
  33. Holleran D, Spohn C (2004) On the use of the total incarceration variable in sentencing research. Criminology 42(1):211–240CrossRefGoogle Scholar
  34. Johnson BD (2006) The Multilevel context of criminal sentencing: integrating judge and county level influences in the study of courtroom decision making. Criminology 44(2):259–298CrossRefGoogle Scholar
  35. Kautt PM (2002) Location, location, location: interdistrict and intercircuit variation in sentencing outcomes for federal drug-trafficking offenses. Justice Q 19:633–671CrossRefGoogle Scholar
  36. Keil TJ, Vito GF (1989) Race, homicide severity, and application of the death penalty: a consideration of the Barnett scale. Criminology 27:511–536CrossRefGoogle Scholar
  37. Kingsnorth R, Lopez J, Wentworth J (1998) Adult sexual assault: the role of racial/ethnic composition in prosecution and sentencing. J Crim Justice 26:359–371CrossRefGoogle Scholar
  38. Kingsnorth R, MacIntosh R, Sutherland S (2002) Criminal charge or probation violation: Prosecutorial discretion and implications for research. Criminology 40:553–578CrossRefGoogle Scholar
  39. Kingsnorth R, MacIntosh R, Wentworth J (1999) Sexual assault: the role of prior relationship and victim characteristics in case processing. Justice Q 16:275–302CrossRefGoogle Scholar
  40. Klepper S, Nagin D, Tierney LJ (1983) Discrimination in the criminal justice system: a critical appraisal of the literature. In: Blumstein A, Cohen J, Martin S, Tonry M (eds) Research on sentencing: the search for reform, vol II. National Academy Press, Washington, DCGoogle Scholar
  41. Kurlychek M, Johnson BD (2004) The juvenile penalty: a comparison of juvenile and young adult sentencing outcomes. Criminology 42:485–517CrossRefGoogle Scholar
  42. Leiber M, Mack K (2003) The individual and joint effects of race, gender, and family status on juvenile justice decision making. J Res Crime Delinq 40:34–71CrossRefGoogle Scholar
  43. Leiber M, Jamieson K (1995) Race and decision making within juvenile justice: the importance of context. J Quantitative Criminol 11:363Google Scholar
  44. Leung SF, Yu S (1996) On the choice between sample selection and two-part models. J Econometrics 72:197–229CrossRefGoogle Scholar
  45. Leung SF, Yu S (2000) Collinearity and two-step estimation of sample selection models: problems, origins, and remedies. Comput Econ 15:173–199CrossRefGoogle Scholar
  46. Loeffler, Charles (2006) Using inter-judge sentencing disparity to estimate the effect of imprisonment on criminal recidivism. Harvard University Working PaperGoogle Scholar
  47. Lundman R, Kaufman R (2003) Driving while black: effects of race, ethnicity, and gender on citizen self-reports of traffic stops and police actions. Criminology 41:195–220CrossRefGoogle Scholar
  48. MacMillan R (2000) Adolescent victimization and income deficits in adulthood: rethinking the costs of criminal violence from a life-course perspective. Criminology 38:553–587CrossRefGoogle Scholar
  49. Maddala GS (1985) A survey of the literature on selectivity bias as it pertains to health care markets. Adv Health Econ Health Serv Res 6:3–18Google Scholar
  50. Maxwell C, Garner J, Fagan J (2002) The preventative effects of arrest on intimate partner violence. Criminol Publ Policy 2:51–80CrossRefGoogle Scholar
  51. McCarthy B, Smith BL (1986) The conceptualization of discrimination in the juvenile justice process: the impact of administrative factors and screening decisions on juvenile court dispositions. Criminology 24:41–64CrossRefGoogle Scholar
  52. Messner SF, Deane G, Beaulieu M (2002) A log-multiplicative association model for allocating homicides with unknown victim–offender relationships. Criminology 40:457–479CrossRefGoogle Scholar
  53. Messner SF, Deane GD, Anselin L, Pearson-Nelson B (2005) Locating the vanguard in rising and falling homicide rates across US cities. Criminology 43(3):661–696CrossRefGoogle Scholar
  54. Miethe TD, Moore CA (1985) Socioeconomic disparities under determinant sentencing systems: a comparison of preguideline and postguideline practices in Minnesota. Criminology 23(2):337–363CrossRefGoogle Scholar
  55. Moffitt R (1999) Econometric methods for labor market analysis. In: Ashenfelter O, Card D (eds) Handbook of labor economics, vol 3A. North-Holland, AmsterdamGoogle Scholar
  56. Myers MA (1988) Social background and the sentencing behavior of judges. Criminology 26:649–675CrossRefGoogle Scholar
  57. Myers MA, Talarico SM (1986) Urban justice, rural injustice? Urbanization and its effects on sentencing. Criminology 24:367–391CrossRefGoogle Scholar
  58. Newey WK (1999) Consistency of two step sample selection estimators despite misspecifications of distribution. Econ Lett 63:129–132CrossRefGoogle Scholar
  59. Nobiling T, Spohn C, Delone M (1998) A tale of two counties: unemployment and sentence severity. Justice Q 15:459–485CrossRefGoogle Scholar
  60. Osgood DW, Finken L, McMorris B (2002) Analyzing multiple item measures of crime and deviance II: Tobit regression analysis of transformed scores. J Quantitative Criminol 18:859–866Google Scholar
  61. Owens, EG (2006) More time, less crime? The incapacitative effect of sentence enhancements. University of Maryland working paperGoogle Scholar
  62. Peterson RD, Hagan J (1984) Changing conceptions of race: toward an account of anomalous findings of sentencing research. Am Sociol Rev 49:56–70CrossRefGoogle Scholar
  63. Puhani PA (2000) The Heckman correction for sample selection and its critique. J Econ Surveys 14:53–68CrossRefGoogle Scholar
  64. Reitz K (1998) Modeling discretion in American sentencing systems. Law & Policy 20:389–428CrossRefGoogle Scholar
  65. Rhodes W (1991) Federal criminal sentencing: some measurement issues with application to pre-guideline sentencing disparity. J Criminal Law Criminol 81:1002CrossRefGoogle Scholar
  66. Robertson AA, Grimes PW, Rogers KE (2002) A short-run cost-benefit analysis of community based interventions for juvenile offenders. Crime Delinq 47:265–284CrossRefGoogle Scholar
  67. Smith DA (1986) The plea bargaining controversy. J Criminal Law Criminol 77:949–967CrossRefGoogle Scholar
  68. Smith DA, Brame R (2003) Tobit models in social science research: some limitations and a more general alternative. Sociol Methods Res 31:364–388CrossRefGoogle Scholar
  69. Sorenson J, Wallace DH (1999) Prosecutorial discretion in seeking death: an analysis of racial disparity in the pretrial stages of case processing in a Midwestern county. Justice Q 16:559–578CrossRefGoogle Scholar
  70. Spohn C, Holleran D (2002) The effect of imprisonment on recidivism rates of felony offenders: a focus on drug offenders. Criminology 40:329–358CrossRefGoogle Scholar
  71. Spohn C, Horney J (1996) The impact of rape law reform on the processing of simple and aggravated rape cases. J Criminal Law Criminol 86:861–886 CrossRefGoogle Scholar
  72. Stata Corp. (2003) Stata Statistical Software Release 8.0. Reference Manual G-M. Stata Corporation, College Station, TX, p 71Google Scholar
  73. Steen S, Engen RL, Gainey RR (2005) Images of danger and culpability: racial stereotyping, case processing, and criminal sentences. Criminology 43:435–467CrossRefGoogle Scholar
  74. Steffensmeier D, Kramer JH, Streifel C (1993) Gender and imprisonment decisions. Criminology 31:411–446CrossRefGoogle Scholar
  75. Steffensmeier D, Ulmer JT, Kramer JH (1998) The interaction of race, gender, and age in criminal sentencing: the punishment cost of being young, black, and male. Criminology 36:763–798CrossRefGoogle Scholar
  76. Steffensmeier D, DeMuth S (2001) Ethnicity and judges’ sentencing decisions: Hispanic-Black-White comparisons. Criminology 39:145–176CrossRefGoogle Scholar
  77. Stolzenberg RM, Relles DA (1990) Theory testing in a world of constrained research design: the significance of Heckman’s censored sampling bias correction for nonexperimental research. Sociol Methods Res 18:395–415CrossRefGoogle Scholar
  78. Stolzenberg RM, Relles DA (1997) Tools for intuition about sample selection bias and its correction. Am Sociol Rev 62:494–507CrossRefGoogle Scholar
  79. Stolzenberg L, D’Alessio SJ, Eitle D (2004) A multilevel test of racial threat theory. Criminology 42:673–698CrossRefGoogle Scholar
  80. Tibbets SG, Piquero AR (1999) The influence of gender, low birth weight, and disadvantaged environment in predicting early onset of offending: a test of Moffitt’s interactional hypothesis. Criminology 37:843–878CrossRefGoogle Scholar
  81. Tobin J (1958) Estimation of relationships for limited dependent-variables. Econometrica 26:24–36CrossRefGoogle Scholar
  82. Ulmer JT (2000) The rules have changed—so proceed with caution: a comment on Engen and Gainey’s methods for modeling sentencing outcomes under guidelines. Criminology 38:1231–1243CrossRefGoogle Scholar
  83. Ulmer JT, Johnson BD (2004) Sentencing in context: a multilevel analysis. Criminology 42:137–178CrossRefGoogle Scholar
  84. Ulmer JT, Kramer JH (1996) Court communities under sentencing guidelines: dilemmas of formal rationality and sentencing disparity. Criminology 34:383–408CrossRefGoogle Scholar
  85. Vella F (1999) Estimating models with sample selection bias: a survey. J Hum Resour 33:127–169CrossRefGoogle Scholar
  86. Wheeler S, Weisburd D, Bode N (1982) Sentencing the white collar offender: rhetoric and reality. Am Sociol Rev 47:641–659CrossRefGoogle Scholar
  87. Winship C, Mare RD (1992) Models for selection bias. Ann Rev Sociol 18:327–350CrossRefGoogle Scholar
  88. Wooldredge JD (1998) Analytic rigor in studies of disparities in criminal case processing. J Quantitative Criminol 14:155–179CrossRefGoogle Scholar
  89. Wooldridge J (2005) Introductory economics: a modern approach. South Western College PublishingGoogle Scholar
  90. Wooldredge JD, Thistlewaite A (2004) Bilevel disparities in court dispositions for intimate assault. Criminology 42:417–456CrossRefGoogle Scholar
  91. Worrall JL (2002) If you build it, they will come: consequences of improved citizen complaint review procedures. Crime Delinq 48:355–379CrossRefGoogle Scholar
  92. Wright JP, Cullen FT (2000) Juvenile involvement in occupational delinquency. Criminology 38:863–892CrossRefGoogle Scholar
  93. Zatz MS (2000) The convergence of race, ethnicity, gender, and class on court decision making: looking toward the 21st century. Policies, processes, and decisions of the criminal justice system, vol. 3. US Department of Justice, Washington, DC, pp 503–552Google Scholar
  94. Zatz MS, Hagan J (1985) Crime, time, and punishment: an exploration of selection bias in sentencing research. J Quantitative Criminol 1:103–126CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • Shawn Bushway
    • 1
  • Brian D. Johnson
    • 2
  • Lee Ann Slocum
    • 2
  1. 1.School of Criminal JusticeState University of New York at AlbanyAlbanyUSA
  2. 2.Department of Criminology and Criminal JusticeUniversity of MarylandCollege ParkUSA

Personalised recommendations