Abstract
Objectives
Longitudinal data offer many advantages to criminological research yet suffer from attrition, namely in the form of sample selection bias. Attrition may undermine reaching valid inferences by introducing systematic differences between the retained and attrited samples. We explored (1) if attrition biases correlates of recidivism, (2) the magnitude of bias, and (3) how well methods of correction account for such bias.
Methods
Using data from the LoneStar Project, a representative longitudinal sample of reentering men in Texas, we examined correlates of recidivism using official measures of recidivism under four sample conditions: full sample, listwise deleted sample, multiply imputed sample, and two-stage corrected sample. We compare and contrast the results regressing rearrest on a range of covariates derived from a pre-release baseline interview across the four sample conditions.
Results
Attrition bias was present in 44% of variables and null hypothesis significance tests differed for the correlates of recidivism in the full and retained samples. The bias was substantial, altering effect sizes for recidivism by a factor as large as 1.6. Neither the Heckman correction nor multiple imputation adequately corrected for bias. Instead, results from listwise deletion most closely mirrored the results of the full sample with 89% concordance.
Conclusions
It is vital that researchers examine attrition-based selection bias and recognize the implications it has on their data when generating evidence of theoretical, policy, or practical significance. We outline best practices for examining the magnitude of attrition and analyzing longitudinal data affected by sample selection.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Other approaches include Manski bounds, Cosslett’s selection model, Newey’s series estimator, Powell’s two-step semiparametric estimator, Robinson’s estimator, along with other statistical methods such as xtARGLS, support vector regression, cluster-based estimating, and kernel mean matching (Winship and Mare 1992). Many of these models with limited dependent variables are estimated using LIMDEP’s statistical software package.
There are circumstances, however, in which reincarceration may actually make someone easier to locate and study as they are in custody (Fahmy et al. 2019). But, overall, it is more difficult to make inferences about this group than those in the general population.
In some cases, researchers rely on propensity score matching (PSM) to correct for bias in their samples (Dehejia and Wahba 2002), though this method is not relevant here and is beyond the scope of the paper. PSM matches subjects based on relevant variables and should reduce the potential bias due to confounding variables (Rosenbaum and Rubin 1983). Although some researchers suggest that PSM holds promise over a listwise deleted sample (Dehejia and Wahba 1998; Lennox et al. 2012), the technique requires a strict set of assumptions that are difficult to meet in many cases (Dehejia 2005), it may not be appropriately estimated, and does not provide a universal correction for selection (Dehejia 2005; Smith and Todd 2005). For instance, PSM’s conditional independence assumption is predicated on the idea that assignment in the treatment group (e.g., retained versus attrited) is based on relevant observed characteristics (Campbell et al. 2020; Tucker 2010). Because of this assumption, however, PSM cannot account for the hidden bias created by unobserved characteristics (Tucker 2010; Wolfolds and Siegel 2019). When the goal is to control for endogeneity that arises from unobservable characteristics (such as in our case), the Heckman selection model is superior (Lennox et al. 2012).
Although the Heckman two-step correction can be completed in two stages, the preferred method involves a Full Information Maximum Likelihood (FIML) model which estimates the equations simultaneously to reduce model error.
We used the following search terms in a Boolean fashion: Heckman AND (“two-step” OR “two-stage” OR “Two-step correction” OR “Two-stage correction” OR “two step” OR “ two stage” OR “Two step correction” OR “Two stage correction” OR “selection” OR “correction” OR “Mills ratio” OR “Mill's ratio” OR “rho”) NOT "two-stage least squares" and searched the following journals: Criminology, Criminal Justice and Behavior, Journal of Developmental and Life-Course Criminology, Journal of Experimental Criminology, Justice Quarterly, Journal of Quantitative Criminology, and Journal of Research in Crime and Delinquency. The initial search resulted in 67 articles. After careful review of the articles for relevance and the use of a Heckman correction, 46 articles were removed, resulting in a final sample of 21 articles.
Although we understand that Little’s (1988) Missing Completely at Random (MCAR) test does not consider unobservable data and cannot determine whether missing data is truly MCAR, we ran the test to help justify our decision to listwise delete missing data. The test statistic was non-significant (p = 1.00), which provides support for our decision. Additionally, only 1.37% of cells were missing, which gives us confidence that that the missing data were sparse enough to utilize listwise deletion. For variables that were missing more than one response, regressions were estimated with a dummy variable adjustment. Coefficients were compared between the mean/mode replaced models and the listwise deleted models and no statistically significant differences existed.
In response to a reviewer’s comment, we have run additional analyses and created a table explicating the use of our exclusion restrictions from the LoneStar Project’s metadata. We closely examined the findings of linear probability models comparing our exclusion restrictions’ ability to predict wave 3 retention versus rearrest from the Clark et al. (2020) paper and are confident that our exclusion restrictions are appropriately justified (see Appendix 2).
We also created a continuous measure, arrest count, representing the number of times a respondent was arrested after release from prison. This estimate was necessary for one modeling strategy—the TPM.
Although probit models are statistically available for the analysis of a binary outcome, in order for us to compare coefficients across models, as suggested by a reviewer, LPMs were the most appropriate.
Consistent with traditional Heckman models, we attempted to use a continuous measure of arrest. Due to the overdispersion of zeros indicating no arrests in our data, this measure was not normally distributed. A heckpoisson command exists in Stata, but our data did not meet poisson distribution assumptions. Given the challenges with normality, a heckprobit was also assessed in our analyses; however, we ultimately decided to run a FIML LPM Heckman model with a binary outcome in order to compare coefficients across models. Therefore, we estimate a FIML LPM Heckman in order to compare equality of coefficients as well as a heckprobit to maintain the integrity of the Heckman correction using a binary outcome. As demonstrated by Tables 4 and 5, statistically significant coefficients did not vary between Heckman models.
It is possible that this variation is due to the loss of analytical power between the full and retained sample. It becomes more difficult to detect statistically significant differences when a sample changes from 791 to 506 people.
These estimates were calculated by first coding each variable to determine how many of the two coefficients across each model were significant and noting the agreement between those coefficients. For each variable across models, if both coefficients were significant or if both coefficients were non-significant, it was coded as 2 (i.e., “agreement”). If one coefficient was significant and the other was non-significant, it was coded as 0 (i.e., “disagreement”). This step is conducted at the variable level, so each variable within the model had an agreement estimate, which were later summed. Step two requires calculating the percent agreement between the models which involved summing agreement/disagreement estimates from each variable and dividing them by the total number of coefficients across models. That equation was \(Model Agreement=\frac{Sum\,of\,Agreement}{2\times Number\,of\,Variables}\times100\).
We are aware that coefficient comparisons, similar to Paternoster et al. (1998), are designed for and assume independent samples. That is not the case for our data. However, this was the most viable way of comparing coefficients across models since we are not able to estimate seemingly unrelated estimation (SUR). SUR is implausible because equations have to be balanced in terms of number of observations, the sureg command in Stata does not allow for conditional statements (such as weights) or use of the same dependent variable, and the SUR method is used to model parameters of all equations simultaneously; thus, we have no way of fitting different model types. Due to these limitations, please use caution when interpreting the findings.
Stolzenberg and Relles (1997) require a continuous outcome variable. Although this may prohibit its use for some research questions, we encourage researchers to move away from binary outcomes which limit the variation and restrain the social world to a binary.
References
Allison PD (2000) Multiple imputation for missing data: a cautionary tale. Soc Methods Res 28(3):301–309
Allison PD (2002) Missing data. Sage, Thousand Oaks
Alper M, Durose MR, Markman J (2018) 2018 Update on prisoner recidivism: a 9-year follow-up period (2005–2014). U.S. Department of Justice, Bureau of Justice Statistics, Washington, DC
Anderson B (2017) Selection models and weak instruments. Musings. Retrieved from https://info.umkc.edu/drbanderson/selection-models-and-weak-instruments/
Asendorpf JB, Van De Schoot R, Denissen JJA, Hutteman R (2014) Reducing bias due to systematic attrition in longitudinal studies: the benefits of multiple imputation. Int J Behav Dev 38(5):453–460. https://doi.org/10.1177/0165025414542713
Badawi MA, Eaton WW, Myllyluoma J, Weimer LG, Gallo J (1999) Psychopathology and attrition in the Baltimore ECA 15-year follow-up 1981–1996. Soc Psychiatry Psychiatr Epidemiol 34(2):91–98. https://doi.org/10.1007/s001270050117
Barber J, Kusunoki Y, Gatny H, Schulz P (2016) Participation in an intensive longitudinal study with weekly web surveys over 2.5 years. Jl Med Internet Res 18(6):1–12. https://doi.org/10.2196/jmir.5422
Barrett DE, Katsiyannis A, Zhang D, Zhang D (2014) Delinquency and recidivism: a multicohort, matched-control study of the role of early adverse experiences, mental health problems, and disabilities. J Emot Behav Disord 22(1):3–15. https://doi.org/10.1177/1063426612470514
Barry AE (2005) How attrition impacts the internal and external validity of longitudinal research. J Sch Health 75(7):267–270. https://doi.org/10.1111/j.1746-1561.2005.00035.x
Bascle G (2008) Controlling for endogeneity with instrumental variables in strategic management research. Strateg Organ 6(3):285–327. https://doi.org/10.1177/1476127008094339
Beijersbergen KA, Dirkzwager AJE, Nieuwbeerta P (2016) Reoffending after release: Does procedural justice during imprisonment matter? Crim Justice Behav 43(1):63–82. https://doi.org/10.1177/0093854815609643
Berg MT, Huebner BM (2011) Reentry and the ties that bind: an examination of social ties, employment, and recidivism. Justice Q 28(2):382–410. https://doi.org/10.1080/07418825.2010.498383
Berk RA (1983) An introduction to sample selection bias in sociological data. Am Sociol Rev 48(3):386–398
Bolanos F, Herbeck D, Christou D, Lovinger K, Pham A, Raihan A et al (2012) Using Facebook to maximize follow-up response rates in a longitudinal study of adults who use methamphetamine. Subst Abuse Res Treat 6:1–11
Boys A, Marsden J, Stillwell G, Hatchings K, Griffiths P, Farrell M (2003) Minimizing respondent attrition in longitudinal research: practical implications from a cohort study of adolescent drinking. J Adolesc 26(3):363–373
Brame R, Paternoster R (2003) Missing data problems in criminological research: two case studies. J Quant Criminol 19(1):55–78. https://doi.org/10.4324/9781315089256-5
Brame R, Piquero AR (2003) Selective attrition and the age-crime relationship. J Quant Criminol 19(2):107–127
Breen R, Karlson KB, Holm A (2018) Interpreting and understanding logits, probits, and other nonlinear probability models. Ann Rev Sociol 44(1):39–54. https://doi.org/10.1146/annurev-soc-073117-041429
Bushway S, Johnson BD, Slocum LA (2007) Is the magic still there? The use of the Heckman two-step correction for selection bias in criminology. J Quant Criminol 23(2):151–178
Butler J, Quinn SC, Fryer CS, Garza MA, Kim KH, Thomas SB (2013) Characterizing researchers by strategies used for retaining minority participants: Results of a national survey. Contemp Clin Trials 36(1):61–67
Butler HD, Steiner B, Makarios MD, Travis LF (2017) Assessing the effects of exposure to supermax confinement on offender postrelease behaviors. Prison J 97(3):275–295. https://doi.org/10.1177/0032885517703925
Campbell DT, Stanley JC (1963) Experimental and quasi-experimental designs for research. Rand McNally, Chicago
Campbell CM, Labrecque RM, Weinerman M, Sanchagrin K (2020) Gauging detention dosage: assessing the impact of pretrial detention on sentencing outcomes using propensity score modeling. J Crim Just 70:1–14. https://doi.org/10.1016/j.jcrimjus.2020.101719
Carkin DM, Tracy PE (2015) Adjusting for unit non-response in surveys through weighting. Crime Delinq 61(1):143–158. https://doi.org/10.1177/0011128714556739
Cartwright N (2007) Are RCTs the gold standard? BioSocieties 2(1):11–20. https://doi.org/10.1017/s1745855207005029
Cattaneo MD (2010) Efficient semiparametric estimation of multi-valued treatment effects under ignorability. J Econom 155(2):138–154. https://doi.org/10.1016/j.jeconom.2009.09.023
Certo ST, Busenbark JR, Woo H-S, Semadeni M (2016) Sample selection bias and Heckman models in strategic management research. Strateg Manag J 37:2639–2657. https://doi.org/10.1002/smj
Cerulli G (2014) Ivtreatreg: a command for fitting binary treatment models with heterogeneous response to treatment and unobservable selection. Stata J 14(3):453–480. https://doi.org/10.1177/1536867x1401400301
Chang MW, Brown R, Nitzke S (2009) Participant recruitment and retention in a pilot program to prevent weight gain in low-income overweight and obese mothers. BMC Public Health 9:1–11. https://doi.org/10.1186/1471-2458-9-424
Chatfield MD, Brayne CE, Matthews FE (2005) A systematic literature review of attrition between waves in longitudinal studies in the elderly shows a consistent pattern of dropout between differing studies. J Clin Epidemiol 58(1):13–19. https://doi.org/10.1016/j.jclinepi.2004.05.006
Clark VA (2016) Predicting two types of recidivism among newly released prisoners: first addresses as “launch pads” for recidivism or reentry success. Crime Delinq 62(10):1364–1400. https://doi.org/10.1177/0011128714555760
Clark VA, Duwe G (2018) From solitary to the streets: the effect of restrictive housing on recidivism. Corrections. https://doi.org/10.1080/23774657.2017.1416318
Clark KJ, Mitchell MM, Fahmy C, Pyrooz DC, Decker SH (2020) What if they are all high-risk for attrition? Correlates of retention in a longitudinal study of reentry from prison. Int J Offender Ther Comp Criminol. https://doi.org/10.1177/0306624X20967934
Claus RE, Kindleberger LR, Dugan MC (2002) Predictors of attrition in a longitudinal study of substance abusers. J Psychoact Drugs 34(1):69–74
Coen AS, Patrick DC, Shern DL (1996) Minimizing attrition in longitudinal studies of special populations: an integrated management approach. Eval Program Plan 19(4):309–319
Cordray S, Polk K (1983) The implications of respondent loss in panel studies of deviant behavior. J Res Crime Delinq 20(2):214–242. https://doi.org/10.1177/002242788302000205
Costanza SE, Cox SM, Kilburn JC (2015) The impact of halfway houses on parole success and recidivism. J Sociol Res 6(2):39–55. https://doi.org/10.5296/jsr.v6i2.8038
Crisanti AS, Case BF, Isakson BL, Steadman HJ (2014) Understanding study attrition in the evaluation of jail diversion programs for persons with serious mental illness or co-occurring substance use disorders. Crim Justice Behav 41(6):772–790
Cullen FT, Pratt TC, Graham A (2019) Why longitudinal research is hurting criminology. The Criminologist 5:63
Curtis R (2010) Getting good data from people that do bad things: Effective methods and techniques for conduting research with hard-to-reach and hidden populations. In: Bernasco W (ed) Offenders on offending: learning about crime from criminals. Willan Publishing, London, pp 141–158
Daquin JC, Daigle LE, Listwan SJ (2016) Vicarious victimization in prison: examining the effects of witnessing victimization while incarcerated on offender reentry. Crim Justice Behav 43(8):1018–1033
David MC, Alati R, Ware RS, Kinner SA (2013) Attrition in a longitudinal study with hard-to-reach participants was reduced by ongoing contact. J Clin Epidemiol 66(5):575–581
Davidov E, Yang-Hansen K, Gustafsson JE, Schmidt P, Bamberg S (2006) Does money matter? A theory-driven growth mixture model to explain travel-mode choice with experimental data. Methodology 2(3):124–134. https://doi.org/10.1027/1614-2241.2.3.124
Davidson R, MacKinnon JG (1993) Estimation and inference in econometrics. Oxford University Press, New York
Deeg DJH, Van Tilburg T, Smit JH, De Leeuw ED (2002) Attrition in the longitudinal aging study Amsterdam: the effect of differential inclusion in side studies. J Clin Epidemiol 55(4):319–328. https://doi.org/10.1016/S0895-4356(01)00475-9
Dehejia RH (2005) Does matching overcome LaLonde’s critique of non-experimental estimators? A postcript. In: Nber working paper series
Dehejia RH, Wahba S (1998) Causal effects in non-experimental studies: re-evaluating the evaluation of training programs (No. 6586)
Dehejia RH, Wahba S (2002) Propensity score-matching methods for nonexperimental causal studies. Rev Econ Stat 84(1):151–161
Duan N, Manning WG, Morris CN, Newhouse JP (1983) A comparison of alternative models for the demand for medical care. J Bus Econ Stat 1(2):115–126. https://doi.org/10.1080/07350015.1983.10509330
Eidson JL, Roman CG, Cahill M (2016) Successes and challenges in recruiting and retaining gang members in longitudinal research: lessons learned from a multisite social network study. Youth Violence Juvenile Just. https://doi.org/10.1177/1541204016657395
Fahmy C, Clark K, Mitchell MM, Decker SH, Pyrooz DC (2019) Method to the madness: tracking and interviewing respondents in a longitudinal study of prisoner reentry. Sociol Methods Res. https://doi.org/10.1177/0049124119875962
Field A (2009) Discovering statistics using SPSS. Sage, Thousand Oaks
Fitzgerald J, Gottschalk P, Moffitt R (1998) An analysis of sample attrition in panel data: the Michigan Panel Study of income dynamics. Population 33(2):251–299
Fumagalli L, Laurie H, Lynn P (2013) Experiments with methods to reduce attrition in longitudinal surveys. J R Stat Soc A Stat Soc 176(2):499–519
Gendreau P, Little T, Goggin C (1996) A meta-analysis of the predictors of adult offender recidivism: What works! Criminology 34(4):575–607
Goodman JS, Blum TC (1996) Assessing the non-random sampling effects of subject attrition in longitudinal research. J Manag 22(4):627–652. https://doi.org/10.1016/s0149-2063(96)90027-6
Greene WH (2011) Econometric analysis, 7th edn. Prentice Hall, Upper Saddle River
Gustavson K, Von Soest T, Karevold E, Roysamb E (2012) Attrition and generalizability in longitudinal studies: findings from a 15-year population-based study and a Monte Carlo simulation study. BMC Public Health 12:918–929. https://doi.org/10.1186/1471-2458-12-918
Hansen WB, Tobler NS, Graham JW (1990) Attrition in substance abuse prevention research: a meta-analysis of 85 longitudinally followed cohorts. Eval Rev 14(6):677–685
Heckman JJ (1976) The common structure of statistical models of truncation, sample selection and limited dependent variables and a simple estimator for such models. Ann Econ Soc Meas 5(4):475–492
Heckman JJ (1979) Sample selection bias as a specification error. Econometrica 47(1):153–161. https://doi.org/10.1007/S10021-01
Hirschi T, Gottfredson M (1983) Age and the explanation of crime. Am J Sociol 89(3):552–584
Hsieh M-L, Hamilton Z, Zgoba KM (2018) Prison experience and reoffending: exploring the relationship between prison terms, institutional treatment, infractions, and recidivism for sex offenders. Sex Abuse 30(5):556–575. https://doi.org/10.1177/1079063216681562
Huebner BM, Varano SP, Bynum TS (2007) Gangs, guns, and drugs: recidivism among serious, young offenders. Criminol Public Policy 6(2):187–222
Katsiyannis A, Whitford DK, Zhang D, Gage NA (2018) Adult recidivism in United States: a meta-analysis 1994–2015. J Child Fam Stud 27(3):686–696. https://doi.org/10.1007/s10826-017-0945-8
Kinner SA (2006) Continuity of health impairment and substance misuse among adult prisoners in Queensland. Australia Int J Prison Health 2(2):101–113. https://doi.org/10.1080/17449200600935711
LaLonde RJ (1986) Evaluating the econometric evaluations of training programs with experimental data. Am Econ Rev 76(4):604–620
Lennox CS, Francis JR, Wang Z (2012) Selection models in accounting research. Account Rev 87(2):589–616. https://doi.org/10.2308/accr-10195
Leung SF, Yu S (1996) On the choice between sample selection and two-part models. J Econom 72:197–229
Liberman AM (2008) The long view of crime: a synthesis of longitudinal research. Springer, Washington, DC
Little RJA (1988) A test of missing completely at random for multivariate data with missing values. J Am Stat Assoc 83(404):1198–1202
Little RJ, Rubin DB (1987) Statistical analysis with missing data. Wiley, New York
Lugo M, Wooldredge J, Pompoco A, Sullivan C, Latessa EJ (2017) Assessing the impact of unit management programs on institutional misconduct and prison “returns.” Just Q. https://doi.org/10.1080/07418825.2017.1357741
Lugtig P (2014) Panel attrition: separating stayers, fast attriters, gradual attriters, and lurkers. Sociol Methods Res 43(4):699–723. https://doi.org/10.1177/0049124113520305
Magruder KM, Ouyang B, Miller S, Tilley BC (2009) Retention of under-represented minorities in drug abuse treatment studies. Clin Trials 6(3):252–260
Malouf ET, Schaefer KE, Witt EA, Moore KE, Stuewig J, Tangney JP (2014) The brief self-control scale predicts jail inmates’ recidivism, substance dependence, and post-release adjustment. Pers Soc Psychol Bull 40(3):334–347. https://doi.org/10.1177/0146167213511666
McGovern V, Demuth S, Jacoby JE (2009) Racial and ethnic recidivism risks: a comparison of postincarceration rearrest, reconviction, and reincarceration among White, Black, and Hispanic releasees. Prison J 89(3):309–327. https://doi.org/10.1177/0032885509339507
McLaughlin P, White N, King K, Hann RG, Williams RJ, Schopflocher D et al (2014) QLS front-line retention manual: methods for achieving a 94% cohort retention rate in longitudinal research
Mears DP, Bales WD (2009) Supermax incarceration and recidivism. Criminology 47(4):1131–1166
Mears DP, Cochran JC (2015) Prisoner reentry in the era of mass incarceration. Sage, Thousand Oaks
Menard S (1995) Applied logistic regression analysis: Sage university series on quantitative applications in the social sciences. Sage, Thousand Oaks
Mitchell MM, McCullough K, Wu J et al. (2018) Survey research with gang and non-gang members in prison: operational lessons from the LoneStar Project. Trends Organ Crim. https://doi.org/10.1007/s12117-018-9331-1
Mitchell MM, Spooner K, Jia D, Zhang Y (2016) The effect of prison visitation on reentry success: a meta-analysis. J Crim Just 47:74–83. https://doi.org/10.1016/j.jcrimjus.2016.07.006
Mutti S, Kennedy RD, Thompson ME, Fong GT (2014) Prepaid monetary incentives-predictors of taking the money and completing the survey: results from the International Tobacco Control (ITC) four-country survey. Sociol Methods Res 43(2):338–355
Odierna DH, Bero LA (2014) Retaining participants in outpatient and community-based health studies: researchers and participants in their own words. SAGE Open 4(4):1–11. https://doi.org/10.1177/2158244014554391
Odierna DH, Schmidt LA (2009) The effects of failing to include hard-to-reach respondents in longitudinal surveys. Am J Public Health 99(8):1515–1521. https://doi.org/10.2105/AJPH.2007.111138
Padfield N, Maruna S (2006) The revolving door at the prison gate: exploring the dramatic increase in recalls to prison. Criminol Crim Just 6(3):329–352. https://doi.org/10.1177/1748895806065534
Paternoster R, Brame R, Mazerolle P, Piquero A (1998) Using the correct statistical test for the equality of regression coefficients. Criminology 36(4):859–866
Peel MJ (2014) Addressing unobserved endogeneity bias in accounting studies: control and sensitivity methods by variable type. Acc Bus Res 44(5):545–571. https://doi.org/10.1080/00014788.2014.926249
Price SM, Park CH, Brenner RA, Lo A, Adams S, Baetz RA, Li T (2016) Participant retention in a longitudinal study: Do motivations and experiences matter? Survey Practice 9(4):1–10. https://doi.org/10.29115/sp-2016-0022
Puhani PA (1997) Foul or fair? The Heckman correction for sample selection and its critique: a short survey. In: ZEW Discussion Papers (No. 97–07). Mannheim, Germany
Puhani PA (2000) The Heckman correction for sample selection and its critique. J Econ Surv 14(1):53–68. https://doi.org/10.1111/1467-6419.00104
Pyrooz DC, Clark KJ, Tostlebe JJ, Decker SH, Orrick E (2021) Gang affiliation and prisoner reentry: discrete-time variation in recidivism by current, former, and non-gang status. J Res Crime Delinq 58(2):192–234. https://doi.org/10.1177/0022427820949895
Ribisl KM, Walton MA, Mowbray CT, Luke DA, Davidson WS, Bootsmiller BJ (1996) Minimizing participant attrition in panel studies through the use of effective retention and tracking strategies: review and recommendations. Eval Program Plann 19(1):1–25
Rosenbaum PR, Rubin DB (1983) The central role of the propensity score in observational studies for causal effects. Biometrika 70(1):41–55. https://doi.org/10.1017/CBO9780511810725.016
Royston P, White IR (2011) Multiple imputation by chained equations (MICE): Implementation in Stata. J Stat Softw 45(4):1–20
Sampson RJ (2010) Gold standard myths: observations on the experimental turn in quantitative criminology. J Quant Criminol 26(4):489–500. https://doi.org/10.1007/s10940-010-9117-3
Schmidt JA, Gruman C, King MB, Wolfson LI (2000) Attrition in an exercise intervention: a comparison of early and later dropouts. J Am Geriatr Soc 48(8):952–960. https://doi.org/10.1111/j.1532-5415.2000.tb06894.x
Shinkfield AJ, Graffam J (2007) Community reintegration of ex-prisoners: type and degree of change in variables influencing successful reintegration. Int J Offender Ther Comp Criminol 53(1):29–42. https://doi.org/10.1177/0306624x07309757
Skeem JL, Manchak S, Peterson JK (2011) Correctional policy for offenders with mental illness: creating a new paradigm for recidivism reduction. Law Hum Behav 35(2):110–126. https://doi.org/10.1007/s10979-010-9223-7
Smith JA, Todd PE (2005) Does matching overcome LaLonde’s critique of nonexperimental estimators? J Econom 125(1–2):305–353. https://doi.org/10.1016/j.jeconom.2004.04.011
Snow DL, Tebes JK, Arthur MW (1992) Panel attrition and external validity in adolescent substance use research. J Consult Clin Psychol 60(5):804–807
Snow WM, Connett JE, Sharma S, Murray RP (2007) Predictors of attendance and dropout at the Lung Health Study 11-year follow-up. Contemp Clin Trials 28(1):25–32. https://doi.org/10.1016/j.cct.2006.08.010
Spohr SA, Suzuki S, Marshall B, Taxman FS, Walters ST (2016) Social support quality and availability affects risk behaviors in offenders. Health Just 4(2):1–10. https://doi.org/10.1186/s40352-016-0033-y
Stevens T, Morash M, Chesney-Lind M (2011) Are girls getting tougher, or are we tougher on girls? Probability of arrest and juvenile court oversight in 1980 and 2000. Just Q 28(5):719–744. https://doi.org/10.1080/07418825.2010.532146
Stolzenberg RM, Relles DA (1990) Theory testing in a world of constrained research design: the significance of Heckman’s censored sampling bias correction for nonexperimental research. Sociol Methods Res 18(4):395–415
Stolzenberg RM, Relles DA (1997) Tools for intuition about sample selection bias and its correction. Am Sociol Rev 62(3):494–507
Tabachnick BG, Fidell LS (2001) Using multivariate statistics, 4th edn. Allyn & Bacon, Needham Heights
Taxman FS (2017) Are you asking me to change my friends? Criminol Public Policy 16(3):775–782. https://doi.org/10.1111/1745-9133.12328
Taylor CJ (2015) Recent victimization and recidivism: the potential moderating effects of family support. Violence Vict 30(2):342–360
Thoits PA (1995) Stress, coping, and social support processes: Where are we? What next? J Health Soc Behav 35:53–79. https://doi.org/10.2307/2626957
Thornberry TP, Bjerregaard B, Miles W (1993) The consequences of respondent attrition in panel studies: a simulation based on the Rochester Youth Development Study. J Quant Criminol 9(2):127–158. https://doi.org/10.1007/BF01071165
Tucker JW (2010) Selection bias and econometric remedies in accounting and finance research. J Account Lit 29:31–57
Uggen C (2000) Work as a turning point in the life course of criminals: a duration model of age, employment, and recidivism. Am Sociol Rev 65(4):529–546
Van Buuren S, Boshuizen HC, Knook DL (1999) Multiple imputation of missing blood pressure covariates in survival analysis. Stat Med 18(6):681–694
Van de Ven W, Van Praag B (1981) The demand for deductibles in private health insurance: a probit model with sample selection. J Econom 17(2):229–252. https://doi.org/10.1016/0304-4076(81)90028-2
Vella F (1998) Estimating models with sample selection bias: a survey. J Hum Resour 33(1):127–169
Visher CA, Travis J (2003) Transitions from prison to community: understanding individual pathways. Ann Rev Sociol 29:89–113. https://doi.org/10.1146/annurev.soc.29.010202.095931
Wadsworth T, Roberts JM (2008) When missing data are not missing: a new approach to evaluating supplemental homicide report imputation strategies. Criminology 46(4):841–870. https://doi.org/10.1111/j.1745-9125.2008.00129.x
Western B (2018) Homeward: life in the year after prison. Russell Sage Foundation, New York
Western B, Braga AA, Kohl R (2017) A longitudinal survey of newly-released prisoners: methods and design of the Boston Reentry Study. Fed Probat 81(1):32–40. https://doi.org/10.3868/s050-004-015-0003-8
Wimberly AS, Engstrom M (2018) Stress, substance use, and yoga in the context of community reentry following incarceration. J Correct Health Care 24(1):96–103. https://doi.org/10.1177/1078345817726536
Winship C, Mare RD (1992) Models for sample selection bias. Ann Rev Sociol 18:327–350
Wolfolds SE, Siegel J (2019) Misaccounting for endogeneity: the peril of relying on the Heckman two-step method without a valid instrument. Strateg Manag J 40(3):432–462. https://doi.org/10.1002/smj.2995
Wong JS, Bouchard J, Gushue K, Lee C (2018) Halfway out: an examination of the effects of halfway houses on criminal recidivism. Int J Offender Ther Comp Criminol. https://doi.org/10.1177/0306624X18811964
Wooldridge JM (2010) Econometric analysis of cross section and panel data, 2nd edn. MIT Press, Cambridge
Young AF, Powers JR, Bell SL (2006) Attrition in longitudinal studies: Who do you lose? Aust N Z J Public Health 30(4):353–361. https://doi.org/10.1111/j.1467-842X.2006.tb00849.x
Young AF, Powers J, Wheway V (2007) Working with longitudinal data: Attrition and retention, data quality, measures of change and other analytical issues. Int J Multiple Res Approach 1(2):175–187
Yu B, Gastwirth JL (2010) How well do selection models perform? Assessing the accuracy of art auction pre-sale estimates. Stat Sin 20(2):837–852
Acknowledgements
Funding was provided by National Institute of Justice (Grant No. 2014-MU-CX-0111).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix 1
See Table 6.
Appendix 2
See Table 7.
Appendix 3
See Table 8.
Appendix 4: Description of Constructs and Coding
Prison visits (mean score of family members and friends visiting); response options: (Never, A few times, Monthly, Bi-weekly, Weekly, Daily).
-
Lead in question: In the last six months, did you have visitation privileges? (No, Yes)
-
[IF YES] Please tell me if you have been visited by the following people:
-
Any family members?
-
Any friends who are not gang members?
-
Delinquent peers (mean score); response options: (No, Yes).
… Think about your closest friends before being incarcerated. A close friend is someone you could call in an emergency, someone you can trust.
Have any of those friends ever…
-
been arrested?
-
been convicted of a crime?
-
been in a correctional facility, such as a jail, prison, or juvenile correctional facility?
-
had problems with drugs or alcohol?
-
Are any of those friends currently in a correctional facility?
Procedural justice (mean score); response options: (Always, Most of the time, Sometimes, Never); alpha = 0.89.
How often do police officers…
-
give people a chance to tell their side of the story before they make decisions?
-
treat people fairly?
-
respect people’s rights?
-
make decisions that are good for everyone in the community?
-
clearly explain the reasons for their actions and decisions?
-
treat people with dignity and respect?
-
try to do what is best for the people they are dealing with?
Low self-control (mean score); response options: (1 = not at all like you, 2 = A little bit like you, 3 = Somewhat like you, 4 = More so like you, 5 = very much like you); alpha = 0.80.
-
You are good at resisting temptation. (reverse coded)
-
You have a hard time breaking bad habits.
-
You say inappropriate things.
-
You do certain things that are bad for you if they are fun.
-
You refuse things that are bad for you. (reverse coded)
-
Pleasure and fun sometimes keeps you from getting work done.
-
You have trouble concentrating.
-
You are able to work effectively toward long-term goals. (reverse coded)
-
Sometimes you can’t stop yourself from doing something, even if you know it is wrong.
-
You often act without thinking through all the alternatives.
-
You have iron self-discipline. (reverse coded)
Stress (mean score); response options: (All of the time, Most of the time, Sometimes, None of the time); alpha = 0.64.
In the past month, how often have you felt…
-
that you were unable to control the important things in your life?
-
confident about your ability to handle your personal problems? (reverse coded)
-
that things were going your way? (reverse coded)
-
difficulties were piling up so high that you could not overcome them?
-
worried or stressed about your upcoming reentry to the community?
Social support (mean score); response options: (Strongly agree, Agree, Disagree, Strongly disagree); alpha = 0.95.
You have someone in your family who…
-
is willing to help you make decisions.
-
really tries to help you.
-
can give you the emotional help and support you need.
-
provide help or advice on finding a place to live.
-
provide help or advice on finding a job.
-
provide support for dealing with a substance abuse problem.
-
provide transportation to work or other appointments if needed.
-
provide financial support.
Social capital (mean score); response options: (Strongly agree, Agree, Disagree, Strongly disagree); alpha = 0.74.
-
You can do just about anything you really set your mind to.
-
You often feel helpless dealing with the problems of life. (reverse coded)
-
You have little control over the things that happen to you. (reverse coded)
-
There is really no way you can solve some of the problems you have. (reverse coded)
-
What happens to you in the future mostly depends on you.
-
You are tired of the problems caused by the crimes you committed.
-
You want to get your life straightened out.
-
You think you will be able to stop committing crimes when released.
-
You will give up friends and hangouts that get you into trouble.
Rights and permissions
About this article
Cite this article
Mitchell, M.M., Fahmy, C., Clark, K.J. et al. Non-random Study Attrition: Assessing Correction Techniques and the Magnitude of Bias in a Longitudinal Study of Reentry from Prison. J Quant Criminol 38, 755–790 (2022). https://doi.org/10.1007/s10940-021-09516-7
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10940-021-09516-7