Measurement Error in Criminal Justice Data

  • John Pepper
  • Carol Petrie
  • Sean Sullivan
Chapter

Abstract

While accurate data are critical in understanding crime and assessing criminal justice policy, data on crime and illicit activities are invariably measured with error. In this chapter, we illustrate and evaluate several examples of measurement error in criminal justice data. Errors are evidently pervasive, systematic, frequently related to behaviors and policies of interest, and unlikely to conform to convenient textbook assumptions. Using both convolution and mixing models of the measurement error generating process, we demonstrate the effects of data error on identification and statistical inference. Even small amounts of data error can have considerable consequences. Throughout this chapter, we emphasize the value of auxiliary data and reasonable assumptions in achieving informative inferences, but caution against reliance on strong and untenable assumptions about the error generating process.

References

  1. Anglin MD, Caulkins JP, Hser Y (1993) Prevalence estimation: policy needs, current status, and future potential. J Drug Issues 23(2):345–360Google Scholar
  2. Azrael D, Cook PJ, Miller M (2001) State and local prevalence of firearms ownership: measurement structure and trends, National Bureau of Economic Research: Working Paper 8570Google Scholar
  3. Bennett T, Holloway K, Farrington D (2008) The statistical association between drug misuse and crime: a meta-analysis. Aggress Violent Behav 13:107–118CrossRefGoogle Scholar
  4. Black D (1970) Production of crime rates. Am Sociol Rev 35:733–48CrossRefGoogle Scholar
  5. Black DA, Berger MC, Scott FA (2000) Bounding parameter estimates with nonclassical measurement error. J Am Stat Assoc 95(451):739–748CrossRefGoogle Scholar
  6. Blumstein A, Rosenfeld R (2009) Factors contributing to U.S. crime trends, in understanding crime trends: workshop report Goldberger AS, Rosenfeld R (eds) Committee on Understanding Crime Trends, Committee on Law and Justice, Division of Behavioral and Social Sciences and Education. The National Academies Press, Washington, DCGoogle Scholar
  7. Bollinger C (1996) Bounding mean regressions when a binary variable is mismeasured. J Econom 73(2):387–99CrossRefGoogle Scholar
  8. Bound J, Brown C, Mathiowetz N (2001) Measurement error in survey data. In: Heckman J, Leamer E (eds) Handbook of econometrics, 5, Ch. 59:3705–3843Google Scholar
  9. Chaiken JM, Chaiken MR (1990) Drugs and predatory crime. Crime Justice: Drugs Crime, 13:203–239CrossRefGoogle Scholar
  10. Dominitz J, Sherman R (2004) Sharp bounds under contaminated or corrupted sampling with verification, with an application to environmental pollutant data. J Agric Biol Environ Stat 9(3):319–338CrossRefGoogle Scholar
  11. Frazis H, Loewenstein M (2003) Estimating linear regressions with mismeasured, possibly endogenous, binary explanatory variables. J Econom 117:151–178CrossRefGoogle Scholar
  12. Frisch R (1934) Statistical confluence analysis by means of complete regression systems. University Institute for Economics, OsloGoogle Scholar
  13. Griliches Z, Hausman JA (1985) Errors in variables in panel data: a note with an example. J Econom 31(1):93–118CrossRefGoogle Scholar
  14. Harrison LD (1995) The validity of self-reported data on drug use. J Drug Issues 25(1):91–111Google Scholar
  15. Harrison L, Hughes A (1997) Introduction – the validity of self-reported drug use: improving the accuracy of survey estimates. In: Harrison L and Hughes A (eds) The validity of self-reported drug use: improving the accuracy of survey estimates. NIDA Research Monograph, vol 167. US Department of Health and Human Services, Rockville, MD, pp 1–16Google Scholar
  16. Horowitz J, Manski C (1995) Identification and robustness with contaminated and corrupted data. Econometrica, 63(2):281–302CrossRefGoogle Scholar
  17. Hotz J, Mullins C, Sanders S (1997) Bounding causal effects using data from a contaminated natural experiment: analyzing the effects of teenage childbearing. Rev Econ Stud 64(4):575–603CrossRefGoogle Scholar
  18. Huber P (1981) Robust statistics. Wiley, New YorkCrossRefGoogle Scholar
  19. Johnston LD, O’Malley PM, Bachman JG (1998) National survey results on drug use from the monitoring the future study, 1975–1997, Volume I: Secondary school students. NIH Publication No. 98-4345. National Institute on Drug Abuse, Rockville, MDGoogle Scholar
  20. Kleck G, Gertz M (1995) Armed resistance to crime: The prevalence and nature of self-defense with a gun. J Crim Law Criminol 86:150–187CrossRefGoogle Scholar
  21. Klepper S, Leamer EE (1984) Consistent sets of estimates for regressions with errors in all variables. Econometrica 52(1):163–183CrossRefGoogle Scholar
  22. Koss M (1993) Detecting the scope of rape: A review of prevalence research methods. J Interpers Violence 8:198–222CrossRefGoogle Scholar
  23. Koss M (1996) The measurement of rape victimization in crime surveys. Crim Justice Behav 23:55–69CrossRefGoogle Scholar
  24. Kreider B, Hill S (2009) Partially identifying treatment effects with an application to covering the uninsured. J Hum Resour 44(2):409–449Google Scholar
  25. Kreider B, Pepper J (2007) Disability and employment: reevaluating the evidence in light of reporting errors. J Am Stat Assoc 102(478):432–441CrossRefGoogle Scholar
  26. Kreider B, Pepper J (2008) Inferring disability status from corrupt data. J Appl Econom 23(3):329–349CrossRefGoogle Scholar
  27. Kreider B, Pepper J. (forthcoming). Identification of expected outcomes in a data error mixing model with multiplicative mean independence. J Business Econ StatGoogle Scholar
  28. Kreider B, Pepper J, Gundersen C, Jolliffee D (2009) Identifying the effects of food stamps on children’s health outcomes when participation is endogenous and misreported. Working PaperGoogle Scholar
  29. Lambert D, Tierney L (1997) Nonparametric maximum likelihood estimation from samples with irrelevant data and verification bias. J Am Stat Assoc 92:937–944CrossRefGoogle Scholar
  30. Lynch JP, Addington LA (eds) (2007) Understanding crime statistics: revisiting the divergence of the NCVS and UCR. Cambridge University Press, CambridgeGoogle Scholar
  31. Lynch J, Jarvis J (2008) Missing data and imputation in the uniform crime reports and the effects on national estimates. J Contemp Crim Justice 24(1):69–85. doi:10.1177/1043986207313028CrossRefGoogle Scholar
  32. Maltz M (1999) Bridging gaps in police crime data: a discussion paper from the BJS Fellows Program Bureau of Justice Statistics, Government Printing Office, Washington, DCGoogle Scholar
  33. Maltz MD, Targonski J (2002) A note on the use of county-level UCR data. J Quant Criminol. 18:297–318CrossRefGoogle Scholar
  34. Manski CF (2007) Identification for prediction and decisions. Harvard University Press, Cambridge, MAGoogle Scholar
  35. Manski CF, Newman J, Pepper JV (2002) Using performance standards to evaluate social programs with incomplete outcome data: general issues and application to a higher education block grant program. 26(4), 355–381Google Scholar
  36. McDowall D, Loftin C (2007) What is convergence, and what do we know about it? In Lynch J, Addington LA (eds) Understanding crime statistics: revisiting the divergence of the NCVS and UCR, Ch. 4: 93–124Google Scholar
  37. McDowall D, Loftin C, Wierseman B (1998) Estimates of the frequency of firearm self-defense from the redesigned national crime victimization survey. Violence Research Group Discussion Paper 20.Google Scholar
  38. Molinari F (2008) Partial identification of probability distributions with misclassified data. J Econom 144(1):81–117CrossRefGoogle Scholar
  39. Mosher CJ, Miethe TD, Phillips DM (2002) The mismeasure of crime. Sage Publications, Thousand Oaks, CAGoogle Scholar
  40. Mullin CH (2005) Identification and estimation with contaminated data: When do covariate Data Sharpen Inference?” J Econom 130:253–272CrossRefGoogle Scholar
  41. National Research Council (2001) Informing America’s policy on illegal drugs: what we don’t know keeps hurting us. Committee on Data and Research for Policy on Illegal Drugs. In: Manski CF, Pepper JV, Petrie CV (eds) Committee on Law and Justice and Committee on National Statistics. Commission on Behavioral and Social Sciences and Education. National Academy Press, Washington, DCGoogle Scholar
  42. National Research Council (2003) Measurement problems in criminal justice research: workshop summary. Pepper JV, Petrie CV. Committee on Law and Justice and Committee on National Statistics, Division of Behavioral and Social Sciences and Education. The National Academies Press, Washington, DCGoogle Scholar
  43. National Research Council (2005) Firearms and violence: a critical review. Committee to improve research information and data on firearms. In: Wellford CF, Pepper JV, Petire CV (eds) Committee on Law and Justice, Division of Behavioral and Social Sciences and Education. The National Academies Press, Washington, DCGoogle Scholar
  44. National Research Council (2008) Surveying victims: Options for conducting the national crime victimization survey. Panel to review the programs of the bureau of justice statistics. In: Groves RM, Cork DL (eds). Committee on National Statistics and Committee on Law and Justice, Division of Behavioral and Social Sciences and Education. The National Academies Press, Washington, DCGoogle Scholar
  45. Office of Applied Studies. (2003). Results from the 2002 National Survey on Drug Use and Health: summary of national finding, (DHHS Publication No. SMA 03-3836, Series H-22). Substance Abuse and Mental Health Services Administration, Rockville, MDGoogle Scholar
  46. Pepper JV (2001) How do response problems affect survey measurement of trends in drug use? In: Manski CF, Pepper JV, Petrie C (eds) Informing America’s policy on illegal drugs: What we don’t know keeps hurting us. National Academy Press, Washington, DC, 321–348Google Scholar
  47. Rand MR, Rennison CM (2002) True crime stories? Accounting for differences in our national crime indicators. Chance 15(1):47–51Google Scholar
  48. Tourangeau R, McNeeley ME (2003) Measuring crime and crime victimization: methodological issues. In: Pepper JV, Petrie CV (eds) Measurement Problems in Criminal Justice Research: Workshop Summary. Committee on Law and Justice and Committee on National Statistics, Division of Behavioral and Social Sciences and Education. The National Academies Press: Washington, DC 0Google Scholar
  49. U.S. Department of Justice (2004) The Nation’s two crime measures, NCJ 122705, http://www.ojp.usdoj.gov/bjs/abstract/ntmc.htm.
  50. U.S. Department of Justice (2006) National Crime Victimization Survey: Criminal Victimization, 2005. Bureau of Justice Statistics Bullentin. http://www.ojp.usdoj.gov/bjs/pub/pdf/cv05.pdf
  51. U.S. Department of Justice (2008) Crime in the United States, 2007-+. Federal Bureau of Investigation, Washington, DC. (table 1). http://www.fbi.gov/ucr/cius2007/data/table_01.html
  52. Wansbeek T, Meijer E (2000) Measurement error and latent variables in econometrics. Elsevier, AmsterdamGoogle Scholar
  53. Wooldridge JM (2002) Econometric analysis of cross section and panel data. MIT Press, Cambridge, MAGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • John Pepper
    • 1
  • Carol Petrie
    • 2
  • Sean Sullivan
    • 1
  1. 1.Department of EconomicsUniversity of VirginiaCharlottesvilleUSA
  2. 2.Committee on Law and JusticeNational Research CouncilWashingtonUSA

Personalised recommendations