Cross-Classified Multilevel Models: An Application to the Criminal Case Processing of Indicted Terrorists


This study provides an application of cross-classified multilevel models to the study of early case processing outcomes for suspected terrorists in U.S. federal district courts. Because suspected terrorists are simultaneously nested within terrorist organizations and criminal court environments, they are characterized by overlapping data hierarchies that involve cross-nested ecological contexts. Cross-classified models provide a useful but underutilized approach for analyzing such data. Using the American Terrorism Study (ATS), this research examines case dismissals, trial adjudications and criminal convictions for a sample of 574 terrorist suspects. Findings indicate that diverse factors affect case processing outcomes, including legal factors such as the number of counts, number of co-defendants, and statute of indictment, extralegal factors such as the ethnicity of the offender, and incident characteristics such as the type of terrorism target. Case processing outcomes also vary significantly across both terrorist groups and criminal courts and are partially explained by select group and court characteristics including the type of terrorist organization and the terrorism trial rate of the court. Results are discussed vis-à-vis contemporary research on terrorism punishments and future directions are suggested for additional applications of cross-classified models in criminological research.

This is a preview of subscription content, access via your institution.

Fig. 1


  1. 1.

    86% of dismissals followed government motions. One case was dismissed due to civil rights violations, two cases were nolle prosequi and five cases were dismissed due to mistrials.

  2. 2.

    To investigate potential selection effects (Bushway et al. 2007), alternative models for trial dispositions and convictions were initially estimated using the subset of cases that were not dismissed (n = 518) along with Heckman’s (1976) two-step selection model (estimated with “heckprob” in Stata 9.2). The results indicated that the selection and substantive equations were statistically independent for both outcomes, suggesting selection bias was not a problem. Although this result raises important questions about the ubiquitous concerns surrounding selection bias in criminal case processing (e.g. Berk 1983), it should be interpreted cautiously. Given the small degree of censoring, the relatively small sample size, the fact that there was not a clear theoretically-defensible exclusion restriction, and concern over the order in which the outcomes occur (e.g. some cases may be dismissed after a plea is entered), results are based on the total sample of n = 574 cases and therefore represent the unconditional estimates for all models reported.

  3. 3.

    A small number of cases (<5%) involved an unusually large number of counts (e.g. 300 or more). Based on the frequency distribution, 50 was selected as the upper cutoff value to prevent these small number of outliers from having undue influence.

  4. 4.

    Included in racketeering offenses are the 35 cases prosecuted explicitly under the new terrorism chapter of the Federal Criminal Code introduced in 1995. These cases were not coded separately because there are relatively few cases prosecuted under terrorism statutes and because no comparable chapter existed prior to 1995. Supplemental analyses were conducted examining the independent effects of these cases and their findings are noted where relevant.

  5. 5.

    Although omission of these measures is unfortunate, it is almost certainly less important for research on early case processing decisions than for research on final sentences. Offense severity is typically the strongest predictor of sentencing, but not necessarily for earlier case processing decisions like those examined in this paper. For instance, Albonetti (1986, 1987) finds little evidence that statutory severity is related to U.S. Attorneys’ decisions to prosecute federal crimes. Some research does suggest that criminal history is related to federal charging behavior (Albonetti 1986; Shermer and Johnson 2010), but the current focus on terrorism suspects largely obviates this concern because the vast majority of convicted terrorists are known to have little or no prior criminal history. For example, 83% of post-guidelines terrorists fall into the lowest criminal history category (Category I) and over 90% fall into either the first or second category (Category I or Category II).

  6. 6.

    All but 8 offenders fell into one of these three categories so other racial/ethnic groups could not be separately examined. Twenty two offenders were also missing information on race/ethnicity and are included in the referent, though supplemental analysis demonstrated that this coding strategy was unrelated to the findings for race/ethnicity.

  7. 7.

    Robust standard errors provide standard error estimates that are adjusted to account for potential violations of model assumptions. Statistical packages such as SAS and STATA provide canned routines for producing cluster-adjusted robust standard errors, as well as other approaches for dealing with clustered data (see Stata Library 2011a).

  8. 8.

    Although the terms hierarchical linear models (HLM) and multilevel models (MLM) are often used interchangeably, they are not necessarily equivalent. Hierarchical models refer to data structures involving exact nesting of lower levels of analysis within higher order units. Multilevel models, on the other hand, represent a broader class of statistical models designed to analyze any nested data structures involving multiple units of analysis. As such, multilevel models encompass hierarchical linear models along with related statistical models, such as the cross-classified models presented here, which are designed to account for nested data that does not follow strict hierarchical ordering.

  9. 9.

    In the classic example of educational research, for instance, students may be nested within both classrooms and within schools, which would represent a three-level hierarchical data structure; however, if students are simultaneously nested with the schools they attend and the neighborhoods in which they live, then the data hierarchy would be cross-classified as in the current research example (see e.g. Goldstein 1994).

  10. 10.

    The total level 2 variance is equal to the sum of terrorist group and district court variances, where var(\( u_{1j} \)) = τu1, var(\( u_{2k} \)) = τu2, and var(\( e_{i(jk)} \)) = σ 2e . If either level 2 variance is zero, then model (2) reduces to the standard two-level hierarchical model. Since the levels are independent (group, district), their variances can be added to the level 1 variance in the intercept-only model to estimate the total variance. For continuous outcomes, this value can then be used to calculate separate intraclass correlations for both group ρu1 = (τu1/(σ 2e  + τu1 + τu2) and district ρu2 = (τu2/(σ 2e  + τu1 + τu2) levels of analysis respectively.

  11. 11.

    The same statistical tests used for random coefficients in standard multilevel models can be applied to cross-classified models. In HLM these include a likelihood ratio test for the deviance statistics of nested models (that vary in their random parameters) as well as χ2 significance tests for variance components included in the model (Raudenbush and Bryk 2002: 63–65). However, specifying random coefficients in cross-classified data can be particularly time consuming and complicated because the number of available random parameters is multiplied by the cross-classification. Ultimately, specification of random effects should be driven by theoretical considerations and tested independently in each nested structure before specifying the full cross-nested random effects (Hox 2002: 136).

  12. 12.

    The model could be further expanded to include cross-level interactions by incorporating level-2 variables (leftwing and/or caseload) in the model equations for individual predictors, (\( \beta_{1(jk)} \)) and (\( \beta_{2(jk)} \)), or by including cross-hierarchy terms to allow the effects of level 2 predictors at one hierarchy to vary across the other. To prevent the sample model from becoming overly complicated, these additional effects are not presented in Eq. 5, but they are briefly discussed here to help illustrate the full versatility and complexity of cross-classified models.

  13. 13.

    The proportion of cases that eventuate in trial differs from the average trial rate across districts because the first is averaged across all 574 indictees and the latter is averaged across the 52 district courts.

  14. 14.

    All multilevel models are estimated as random intercept models because investigation of theoretically-specified random coefficients produced null results for all models. In part, this likely reflects the relatively small sample size and suggests that future research with larger samples is needed to better investigate the extent to which individual terrorist suspect characteristics exert varying effects across group and district contexts.

  15. 15.

    The coefficient for a black defendant approached but did not exceed marginal levels of significance for this model (p = .11).

  16. 16.

    The terrorism trial rate was omitted from this model because by definition it is related to the outcome. Not surprisingly, when included, it is positively, strongly and significantly related to the individual odds of individual trial disposition (b = 5.21, SE = .73).

  17. 17.

    Crimes prosecuted specifically under terrorism statutes were separated from racketeering crimes in this analysis because unlike for other outcomes, they exerted countervailing effects on case conviction.

  18. 18.

    Comparisons of ordinary logistic regression, clustered robust standard error models, and cross-classified multilevel models in the current data showed that cluster-corrected models produce identical coefficients but larger standard errors than ordinary logistic regression models, whereas the cross-classified models produce larger standard errors but also unique coefficient estimates. For additional information on comparisons of different analytical approaches to clustered data see Stata Library (2011b).

  19. 19.

    Ordinary logistic regression typically employs Maximum Likelihood Estimation (MLE) whereas multilevel models often use some variation of this approach, depending upon the statistical program employed and the model specified. In the current analysis, Stata 11.1 is used to estimate the logistic regressions and HLM 6.02 is used for the cross-classified multilevel models. HLM uses full Penalized Quasi-Likelihood (PQL) to generate parameter estimates, which produces approximate empirical Bayes estimates for the random coefficients, generalized least squares estimates for the level 2 coefficients, and approximate maximum likelihood estimates of the variance and covariance parameters in the model (see Raudenbush and Bryk 2002, for a detailed elaboration).

  20. 20.

    For an interesting empirical example of this type of research see Jayasinghe et al. (2003) who used cross-classified models to assess the overlapping influence of grant reviewers and grant submitters on assessment ratings within academic disciplines.

  21. 21.

    Larger samples would also provide an important opportunity to expand the complexity of the cross-classified model to investigate additional research questions involving such things as cross-level and cross-hierarchy interactions. In the current study, initial attempts were made to examine cross-level interactions between offender race/ethnicity and terrorist group affiliation, but the small sample size precluded estimation of these effects.


  1. Albonetti C (1986) Criminality, prosecutorial screening, and uncertainty: toward a theory of discretionary decision making in felony case processings. Criminology 24:623–644

    Article  Google Scholar 

  2. Albonetti C (1987) Prosecutorial discretion: the effects of uncertainty. Law Soc Rev 21:291–314

    Article  Google Scholar 

  3. Albonetti CA (1990) Race and the probability of pleading guilty. J Quant Criminol 6(3):315–334

    Google Scholar 

  4. Berk R (1983) An introduction to sample selection bias in sociological data. Am Soc Rev 48:386–398

    Google Scholar 

  5. Bradely M, Damhousse K, Smith B (2009) Punishing terrorist: a re-examination of the U.S. federal sentencing in the postguidelines era. Int Crim Justice Rev 19:433–455

    Article  Google Scholar 

  6. Britt C (2000) Social context and racial disparities in punishment decisions. Justice Q 17:707–732

    Article  Google Scholar 

  7. Brunson RK, Miller J (2009) Schools, neighborhoods, and adolescent conflicts: a situational examination of reciprocal dynamics. Justice Q 26(2):183–210

    Google Scholar 

  8. Bushway S, Johnson B, Slocum L (2007) Is the magic still there? The use of the Heckman two-step correction for selection bias in criminology. J Quant Criminol 23:151–178

    Article  Google Scholar 

  9. Byongook M, Zager LJ (2007) Police officers’ attitudes toward citizen support: focus on individual, organizational and neighborhood characteristic factors. Polic Int J Police Strateg Manag 30(3):484–497

    Article  Google Scholar 

  10. Chermak S, Gruenewald J (2006) The media’s coverage of domestic terrorism. Justice Q 23:428–461

    Article  Google Scholar 

  11. Damphousse K, Shields C (2007) The morning after: assessing the effect of major terrorism events on prosecution strategies and outcomes. J Contemp Crim Justice 23:174–194

    Article  Google Scholar 

  12. Damphousse KR, Smith BL (2004) Terrorism and empirical testing: using indictment data to assess changes in terrorist conduct. In: Deflem M (ed) The sociology of crime, law and deviance. Elksevier Ltd., San Diego, CA, pp 75–90

  13. Dixon J (1995) The organizational context of criminal sentencing. Am J Sociol 100:1157–1198

    Article  Google Scholar 

  14. Eisenstein J, Flemming R, Nardulli P (1988) The contours of justice: communities and their courts. University Press of America, New York

    Google Scholar 

  15. Emerson R (1983) Holistic effects in social control decision-making. Law Soc Rev 17:425–456

    Article  Google Scholar 

  16. Gelman A, Hill J (2007) Data analysis using regression and multilevel/hierarchical models. Cambridge University Press, Cambridge

    Google Scholar 

  17. Goldstein H (1994) Multilevel cross-classified models. Sociol Methods Res 22:364–375

    Article  Google Scholar 

  18. Goldstein H (2011) Multilevel statistical models. Wiley series in probability and statistics. Wiley, West Sussex, UK

    Google Scholar 

  19. Gottfredson G, Gottfredson D, Payne A, Gottfredson N (2005) School climate predictors of school disorder: results from a national study of delinquency prevention in schools. J Res Crime Delinq 42(4):412–444

    Article  Google Scholar 

  20. Hartley R, Maddan S, Spohn C (2007) Prosecutorial discretion: an examination of substantial assistance departures in federal crack-cocaine and powder-cocaine cases. Justice Q 24:382–407

    Article  Google Scholar 

  21. Heckman J (1976) The common structure of statistical models of truncation, sample selection and limited dependent variables, and a simple estimator for such models. Ann Econ Soc Meas 5:475–492

    Google Scholar 

  22. Hox J (2002) Multilevel analyses: techniques and applications. Erlbaum, Mahwah, NJ

    Google Scholar 

  23. Jayasinghe U, Marsh H, Bond N (2003) A multilevel cross-classified approach to peer review of grant proposals: the effects of assessor and researcher attributes on assessor ratings. J R Stat Soc 166:279–300

    Article  Google Scholar 

  24. Johnson B (2006) The multilevel context of criminal sentencing: integrating judge- and county-level influences. Criminology 44:259–298

    Article  Google Scholar 

  25. Johnson B (2010) Multilevel analysis in the study of crime and justice. Handb Quant Criminol 6:615–648

    Article  Google Scholar 

  26. Johnson B, Ulmer J, Kramer J (2008) The social context of guidelines circumvention: the case of federal district courts. Criminology 46:737–783

    Article  Google Scholar 

  27. Kautt P (2002) Location, location location: interdistrict and intercircuit variation in sentencing outcomes for federal drug-trafficking offenses. Justice Q 19:633–671

    Article  Google Scholar 

  28. Kirk D (2009) Unraveling the contextual effects on student suspension and juvenile arrest: the independent and interdependent influences of school, neighborhood, and family social controls. Criminology 47(2):479–520

    Article  Google Scholar 

  29. Kreft IGG, De Leeuw J (1998) Introducing multilevel modeling. Sage Publications, Thousand Oaks

  30. Kubrin C, Stewart E (2006) Predicting who reoffends: the neglected role of neighborhood context in recidivism studies. Criminology 44(1):165–197

    Article  Google Scholar 

  31. LaFree G, Dugan L, Korte R (2009) The impact of British counterterrorist strategies on political violence in Northern Ireland: comparing deterrence and backlash models. Criminology 47:17–45

    Article  Google Scholar 

  32. LaFree G, Morris N, Dugan L (2010) Cross-national patterns of terrorism. Br J Criminol 50(4):622–649

    Article  Google Scholar 

  33. Landes WM (1978) An economic study of U.S. aircraft hijackings, 1961–1976. J Law Econ 21:1–31

    Article  Google Scholar 

  34. Luke DA (2004) Multilevel modeling. In: Quantitative applications in the social sciences. Sage, Thousand Oaks, CA

  35. Lum C, Kennedy L, Sherley A (2006) Are counter-terrorism strategies effective?: the results of the campbell systematic review on counter-terrorism evaluation research. J Exp Criminol 2(4):489–516

    Article  Google Scholar 

  36. Myers M, Talarico S (1987) The social contexts of criminal sentencing. Springer, New York

    Google Scholar 

  37. Osgood W, Anderson A (2004) Unstructured socializing and rates of delinquency. Criminology 42(3):519–550

    Article  Google Scholar 

  38. Raudenbush S, Bryk A (2002) Three-level models. In: Hierarchical linear models: applications and data analysis methods. Sage, Thousand Oaks, CA

    Google Scholar 

  39. Raudenbush SW, Liu X-F (2000) Statistical power and optimal design for multisite randomized trials. Psychol Methods 5:199–213

    Article  Google Scholar 

  40. Shermer LON, Johnson BD (2010) Criminal prosecutions: examining prosecutorial discretion and charge reductions in U.S. federal district courts. Justice Q 27:394–430

    Article  Google Scholar 

  41. Shields CA, Damphousse KR, Smith BL (2006) Their day in court: assessing guilty pleas among terrorists. J Contemp Crim Justice 22:261–276

    Article  Google Scholar 

  42. Smith BL (1994) Terrorism in America: pipe bombs and pipe dreams. State University of New York Press, Albany, NY

    Google Scholar 

  43. Smith B, Damphousse KR (1996) Punishing political offenders: the effect of political motive on federal sentencing decisions. Criminology 34:289–322

    Article  Google Scholar 

  44. Smith B, Damphousse KR (1998) Terrorism, politics, and punishment: a test of structural-contextual theory and the ‘liberation hypothesis’. Criminology 36:67–92

    Article  Google Scholar 

  45. Smith B, Damphousse KR (2002) American terrorism study: patterns of behavior, investigation and prosecution of American terrorists, Final Report. U.S. Department of Justice. 1999-IJ-CX-0005

  46. Smith BL, Orvis GP (1993) America’s response to terrorism: an empirical analysis of federal intervention strategies during the 1980s. Justice Q 10:661–681

    Article  Google Scholar 

  47. Smith BL, Damphousse KR, Jackson F, Sellers A (2002) The prosecution and punishment of international terrorists in federal courts: 1980–1998. Criminol Public Policy 1:311–338

    Article  Google Scholar 

  48. Smith BL, Damphousse KR, Yang S, Ginther C (2005) Prosecuting politically motivated offenders: the impact of the terrorist label on criminal case outcomes: terrorism and the modern world. Int J Contemp Soc 42:209–226

    Google Scholar 

  49. Snijders T (2005) Power and sample size in multilevel linear models. In: Everitt BS, Howell DC (eds) Encyclopedia of statistics in behavioral science, vol 3. Wiley, New York, pp 1570–1573

    Google Scholar 

  50. Snijders T, Bosker RJ (1999) Multilevel analysis. Sage, Thousand Oaks, CA

    Google Scholar 

  51. Spohn C, Fornago R (2009) U.S. Attorneys and substantial assistance departures: testing for interprosecutor disparity. Criminology 47:813–846

    Article  Google Scholar 

  52. Spohn C, Gruhl J, Welch S (1987) The impact of ethnicity and gender of defendants on the decision to reject or dismiss felony charges. Criminology 25(1):175–191

    Article  Google Scholar 

  53. Spohn C, Spears J (1996) The effect of offender and victim characteristics on sexual assault case processing decisions. Justice Q 13(4):649–679

    Google Scholar 

  54. Stata Library (2011a) Analyzing correlated (clustered) data. UCLA: academic technology services, statistical consulting group. Accessed 1 June 2011

  55. Stata Library (2011b) What are the some of the methods for analyzing clustered data in Stata? UCLA: academic technology services, statistical consulting group. Accessed 1 June 2011

  56. Stith K, Cabranes J (1998) Fear of judging: sentencing guidelines in the federal courts. University of Chicago Press.

  57. Ulmer JT, Johnson BD (2004) Sentencing in context: a multilevel analysis. Criminology 42:137–177

    Article  Google Scholar 

  58. Ulmer JT, Eisenstein J, Johnson BD (2009) Trial penalties in federal sentencing: extra-guidelines factors and district variation. Justice Q 27:560–592

    Article  Google Scholar 

  59. United States Sentencing Commission (2004) Sourcebook of federal sentencing statistics. United States Sentencing Commission. U.S. Department of Justice

  60. Wilmot KA, Spohn C (2004) Prosecutorial discretion and real-offense sentencing: an analysis of relevant conduct under the federal sentencing guidelines. Crim Justice Pol Rev 15(3):324–343

    Google Scholar 

  61. Wooldredge J (2007) Neighborhood effects on felony sentencing. J Res Crime Delinq 44:238–263

    Article  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Brian D. Johnson.

Additional information

This material is based upon work supported by the Science and Technology Directorate of the U.S. Department of Homeland Security under Grant Award Number 2008-ST-061-ST0004, made to the National Consortium for the Study of Terrorism and Responses to Terrorism (START, The views and conclusions contained in this document are those of the author and should not be interpreted as necessarily representing the official policies, either expressed or implied, of the U.S. Department of Homeland Security or START.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Johnson, B.D. Cross-Classified Multilevel Models: An Application to the Criminal Case Processing of Indicted Terrorists. J Quant Criminol 28, 163–189 (2012).

Download citation


  • Terrorism
  • Prosecution
  • Punishment
  • Cross-classified models
  • Cross-nested models
  • Multilevel models