Journal of Quantitative Criminology

, Volume 16, Issue 3, pp 341–367 | Cite as

Invariance of Measures of Prevention Program Effectiveness: A Replication

  • André B. Rosay
  • Denise C. Gottfredson
  • Todd A. Armstrong
  • Michele A. Harmon


Recent literature has suggested that measures of risk and protective factorsfor delinquency and substance use are not equally reliable or valid acrossgender and ethnic groups and has recommended differentiated programming andculturally specific evaluation methods. Three data sets containing up tofive ethnic groups were used to determine the degree to which risk andprotective factors are equally reliable and valid predictors of drug use anddelinquency across gender and ethnic groups. Congeneric measurement modelsand structural equation models were evaluated to determine if the factorstructures for these measures and their covariances with measures of druguse and delinquency were equivalent across gender and ethnic groups. Half ofthe risk and protective factors included in this analysis were found to beequally reliable across gender and ethnic groups. When controlling forreliability differences, all of the risk and protective factors were foundto predict both drug use and delinquncy for all gender and ethnic groups. Interms of the magnitude of these associations, no substantive differenceswere found in the validity of risk and protective factors for drug use anddelinquency. Differences in the validity of risk and protective factors weremore prevalent for delinquency than for drug use. However, all differenceswere substantively trivial. We conclude that measures of prevention programeffectiveness are invariant across gender and ethnic groups.

ethnic differences gender differences measurement of delinquency measurement of drug use risk and protective factors 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Akaike, H. (1974). A new look at the statistical model identification. IEEE Trans. Automatic Control 19: 716–723.Google Scholar
  2. Akaike, H. (1987). Factor analysis and AIC. Psychometrika 52: 317–332.Google Scholar
  3. Akers, R. L., Krohn, M. D., Lanza-Kaduce, L., and Radosevich, N. (1979). Social learning and deviant behavior: A specific test of a general theory. Am. Sociol. Rev. 44: 636–655.Google Scholar
  4. Alwin, D. F., and Jackson, D. J. (1979). Measurement models for response errors in surveys: Issues and applications. In Schuessler, K. F. (ed.), Sociological Methodology: 1980, Jossey-Bass, San Francisco, pp. 68–119.Google Scholar
  5. Anderson, J. C., and Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychol. Bull. 103: 411–423.Google Scholar
  6. Andrews, D. A., Zinger, I., Hoge, R. D., Bonta, J., Gendreau, P., and Cullen, F. T. (1990). Does correctional treatment work? A clinically relevant and psychologically informed meta-analysis. Criminology 28: 369–404.Google Scholar
  7. Bandura, A. (1977). Social Learning Theory, Prentice Hall, Englewood Cliffs, NJ.Google Scholar
  8. Benson, J. (1987). Detecting item bias in affective scales. Educ. Psychol. Measure. 47: 55–67.Google Scholar
  9. Bentler, P. M., and Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychol. Bull. 88: 588–606.Google Scholar
  10. Bollen, K. A. (1986). Sample size and Bentler and Bonett's nonormed fit index. Psychometrika, 51: 375–377.Google Scholar
  11. Bollen, K. A. (1989). Structural Equations with Latent Variables, Wiley, New York.Google Scholar
  12. Boomsma, A. (1982). The robustness of LISREL against small sample size in factor analysis models. In Jöreskog, K., and Wold, H. (eds.), Systems Under Indirect Observation, Part I, North-Holland, Amsterdam, pp. 149–173.Google Scholar
  13. Bozdogan, H. C. (1987). Model selection and Akaike's information criterion (AIC): The general theory and its analytical extensions. Psychometrika 52: 345–370.Google Scholar
  14. Byram, O. W., and Fly, J. W. (1984). Family structure, race and adolescents' alcohol use: A research note. Am. J. Drug Alcohol Abuse 10: 467–478.Google Scholar
  15. Byrne, B. M. (1988). The Self Description Questionnaire III: Testing for equivalent factorial validity across ability. Educ. Psychol. Measure. 48: 397–406.Google Scholar
  16. Carmines, E. G., and McIver, J. P. (1981). Analyzing models with unobserved variables: Analysis of covariance structures. In Bohrnstedt, G. W., and Borgatta, E. F. (eds.), Social Measurement: Current Issues, Sage, Beverly Hills, CA, pp. 66–115.Google Scholar
  17. Carmines, E. G., and Zeller, R. A. (1979). Reliability and Validity Assessment, Sage, Newbury Park, CA.Google Scholar
  18. Castro, F. G., Maddahian, E., Newcomb, M. D., and Bentler, P. M. (1987). A multivariate model of the determinants of cigarette smoking among adolescents. J. Health Soc. Behav. 28: 273–289.Google Scholar
  19. Catalano, R. F., Morrison, D. M., Wells, E. A., Gillmore, M. R., Iritani, B., and Hawkins, J. D. (1992). Ethnic differences in family factors related to early drug initiation. J. Stud. Alcohol. 53: 208–217.Google Scholar
  20. Cole, D. A., and Maxwell, S. E. (1985). Multitrait-multimethod comparisons across populations: A confirmatory factor analytic approach. Multivar. Behav. Res. 20: 389–417.Google Scholar
  21. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika 16: 297–334.Google Scholar
  22. Gerbing, D. W., and Hunter, J. E. (1982). The metric of the latent variables in the LISRELIV analysis. Educ. Psychol. Measure. 42: 423–427.Google Scholar
  23. Gillmore, M. R., Catalano, R. F., Morrison, D. M., Wells, E. A., Iritani, B., and Hawkins, J. D. (1990). Racial differences in acceptability and availability of drugs and early initiation of substance use. Am. J. Drug Alcohol Abuse 16: 185–206.Google Scholar
  24. Gottfredson, D. C. (1997). School-based crime prevention. In Preventing Crime: What Works, What Doesn't, What's Promising, Office of Justice Programs, Washington, DC.Google Scholar
  25. Gottfredson, D. C., and Koper, C. S. (1996). Race and sex differences in the prediction of drug use. J. Consult. Clin. Psychol. 64: 305–313.Google Scholar
  26. Gottfredson, D. C., and Koper, C. S. (1997). Race and sex differences in the measurement of risk for drug use. J. Quant. Criminol. 13: 325–347.Google Scholar
  27. Gottfredson, D. C., Sealock M. D., and Koper, C. S. (1996). Delinquency. In DiClemente, R. J., Hansen, W. B., and Ponton, L. E. (eds.), Handbook of Adolescent Health Risk Behavior, Plenum, New York, pp. 259–288.Google Scholar
  28. Gottfredson, G. D., and Gottfredson, D. C. (1992). Development and Applications of Theoretical Measures for Evaluating Drug and Delinquency Prevention Programs, Gottfredson Associates, Elliott City, MD.Google Scholar
  29. Gottfredson, M. R., and Hirschi, T. (1990). A General Theory of Crime, Stanford University, Stanord, CA.Google Scholar
  30. Hawkins, J. D., Catalano, R. F., and Miller, J. L. (1992). Risk and protective factors for alcohol and other drug problems in early adulthood: Implications for substance abuse prevention. Psychol. Bull. 112: 64–105.Google Scholar
  31. Hayduk, L. A. (1987). Structural Equation Modeling with LISREL, Johns Hopkins University, Baltimore.Google Scholar
  32. Hayduk, L. A. (1996). Structural Equation Modeling with LISREL: Essentials and Advances, Johns Hopkins Univesity, Baltimore.Google Scholar
  33. Hirschi, T. (1969). Causes of Delinquency, University of California, Berkeley.Google Scholar
  34. Hoelter, J. W. (1983). The analysis of covariance structures. Goodness-of-fit indices. Sociol. Methods Res. 11: 325–344.Google Scholar
  35. Horn, J. L., and McArdle, J. J. (1980). Perspectives on mathematicalystatistical model building (MASMOB) in research on aging. In Poon, L. W. (ed.), Aging in the 1980s: Selected Contemporary Issues in the Psychology of Aging, American Psychological Association, Washington, DC, pp. 503–541.Google Scholar
  36. Howell, J. C., Krisberg, B., Wilson, J. J., and Hawkins, J. D. (1995). A Sourcebook on Serious Violent, and Chronic Juvenile Offenders, Sage, Newbury Park, CA.Google Scholar
  37. James, L. R., Mulaik, S. A., and Brett, J. M. (1982). Causal Analysis: Assumptions, Models and Data, Sage, Beverly Hills.Google Scholar
  38. Jessor, R., and Jessor, S. (1977). Problem Behavior and Psychosocial Development: A Longitudinal Study of Youth, Academic Press, New York.Google Scholar
  39. Jöreskog, K. G., and Sörbom, D. (1981). LISREL5: Analysis of Linear Structural Relations by the Method of Maximum Likelihod, International Educational Services, Chicago.Google Scholar
  40. Jöreskog, K. G., and Sörbom, D. (1989). LISREL7: A Guide to the Program and Applictions, 2d ed., SPSS, Chicago.Google Scholar
  41. Kleinman, P., and Lukoff, I. (1978). Ethnic differences in factors related to drug use. J. Health Soc. Behav. 19: 190–199.Google Scholar
  42. Long, J. S. (1983). Covariance Structure Models: An Introduction to LISREL, Sage, Beverly Hills, CA.Google Scholar
  43. MacCallum, R. C., and Tucker, L. R. (1991). Representing sources of error in the commonfactor model: Implications for theory and practice. Psychol. Bull. 109: 502–511.Google Scholar
  44. Marsh, H. W., Balla, J. R., and McDonald, R. P. (1988). Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychol. Bull. 103: 391–410.Google Scholar
  45. McArdle, J. J. (1986). Latent variable growth within behavior genetic models. Behav. Genet. 16: 163–200.Google Scholar
  46. McDonald, R. P., and Marsh, H. W. (1990). Choosing a multivariate model: Noncentrality and goodness of fit. Psychol. Bull. 107: 247–255.Google Scholar
  47. Newcomb, M. D., and Bentler, P. M. (1986). Substance use and ethnicity: Differential impact of peer and adult models. J. Psychol. 120: 83–95.Google Scholar
  48. Newcomb, M. D., Maddahian, E., Skager, R., and Bentler, P. M. (1987). Substance abuse and psychological risk factors among teenagers: Associations with sex, age, ethnicity, and type of school. Am. J. Drug Alcohol Abuse 13: 413–433.Google Scholar
  49. Nunnally, J. C. (1978). Psychometric Theory, McGraw-Hill, New York.Google Scholar
  50. Orlandi, M. A. (ed.) (1992). Cultural Competence for Evaluators: A Guide for Alcohol and Other Drug Abuse Prevention Practitioners Working with Ethnic/Racial Communities, Office of Substance Abuse Prevention, U.S. Department of Health and Human Services, Rockville, MD.Google Scholar
  51. Piquero, A. R., and Rosay, A. B. (1998).The reliability and validity of Grasmick, et al.'s Self-Control Scale: A comment on Longshore, et al., Criminology 36: 157–173.Google Scholar
  52. Reise, S. P., Widaman, K. F., and Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: two approaches for exploring measurement invariance. Psychol. Bull. 114: 552–566.Google Scholar
  53. Rowe, D. C., Vazsonyi, A. T., and Flannery, D. J. (1994). No more than skin deep: ethnic and racial similarity in developmental process. Psychol. Rev. 101: 396–413.Google Scholar
  54. Sampson, R. J. (1985). Sex differences in self-reported delinquency and official records: A multiple-group structural modeling approach. J. Quant. Criminol. 1: 345–367.Google Scholar
  55. Schwarz, G. (1978). Estimating the dimension of a model. Ann. Stat. 6: 461–464.Google Scholar
  56. Tanaka, J. S. (1987). “How big is enough?” Sample size and goodness-of-fit structural equation models with latent variables. Child Dev. 58: 135–146.Google Scholar
  57. Tanaka, J. S., and Huba, G. J. (1985). A fit index for covariance structure models under arbitrary GLS estimation. Br. J. Math. Stat. Psychol. 38: 197–201.Google Scholar
  58. Tucker, L. R., and Lewis, C. (1973). The reliability coefficient for maximum likelihood factor analysis. Psychometrika 38: 1–10.Google Scholar
  59. Wells, E. A., Morrison, D. M., Gillmore, M. R., Catalano, R. F., Iritani, B., and Hawkins, J. D. (1992). Race differences in antisocial behaviors and attitudes and early initiation of substance use. J. Drug Educ. 22: 115–130.Google Scholar
  60. Wheaton, B. (1988). Assessment of fit in overidentified models with latent variables. In Long, J. S. (ed.), Common Problems/Proper Solutions: Avoiding Error in Quantitative Research, Sage, Newbury Park, CA, pp. 193–225.Google Scholar
  61. Wheaton, B., Muthen, B., Alwin, D., and Summers, G. (1977). Assessing reliability and stability in panel models. In Heise, D. (ed.), Sociological Methodology, Jossey-Bass, San Francisco, pp. 84–136.Google Scholar

Copyright information

© Plenum Publishing Corporation 2000

Authors and Affiliations

  • André B. Rosay
    • 1
  • Denise C. Gottfredson
    • 2
  • Todd A. Armstrong
    • 3
  • Michele A. Harmon
    • 4
  1. 1.Department of Sociology and Criminal JusticeUniversity of DelawareNewark
  2. 2.University of MarylandCollege Park
  3. 3.Arizona State University WestUSA
  4. 4.Westat, IncUSA

Personalised recommendations