Skip to main content
SpringerLink
Log in

Five ethical challenges facing data-driven policing

  • Original Research
  • Published:
AI and Ethics Aims and scope Submit manuscript

Abstract

This paper synthesizes scholarship from several academic disciplines to identify and analyze five major ethical challenges facing data-driven policing. Because the term “data-driven policing” encompasses a broad swath of technologies, we first outline several data-driven policing initiatives currently in use in the United States. We then lay out the five ethical challenges. Certain of these challenges have received considerable attention already, while others have been largely overlooked. In many cases, the challenges have been articulated in the context of related discussions, but their distinctively ethical dimensions have not been explored in much detail. Our goal here is to articulate and clarify these ethical challenges, while also highlighting areas where these issues intersect and overlap. Ultimately, responsible data-driven policing requires collaboration between communities, academics, technology developers, police departments, and policy makers to confront and address these challenges. And as we will see, it may also require critically reexamining the role and value of police in society.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Availability of data and material

Not applicable.

Code availability

Not applicable.

References

  1. Heaven, W.D.: Predictive policing algorithms are racist. They need to be dismantled. MIT Technol. Rev. https://www.technologyreview.com/2020/07/17/1005396/predictive-policing-algorithms-racist-dismantled-machine-learning-bias-criminal-justice/ (2020). (Accessed 15 Feb 2021)

  2. Haskins, C.: Academics Confirm Major Predictive Policing Algorithm is Fundamentally Flawed. Vice.com. (2019). https://www.vice.com/en/article/xwbag4/academics-confirm-major-predictive-policing-algorithm-is-fundamentally-flawed. Accessed 25 May 2021.

  3. Moravec, E.R.: Do Algorithms Have a Place in Policing? The Atlantic. https://www.theatlantic.com/politics/archive/2019/09/do-algorithms-have-place-policing/596851/(2019). (Accessed 15 Feb 2021)

  4. Sturgill, K.: Santa Cruz becomes the first U.S. city to ban predictive policing. Los Angel. Times. https://www.latimes.com/california/story/2020-06-26/santa-cruz-becomes-first-u-s-city-to-ban-predictive-policing (2020). (Accessed 15 Feb 2021)

  5. Burbank, K.: City First in Nation to Ban Predictive Policing, Biometric Surveillance Tech. Sfgate.com. (2021). https://www.sfgate.com/news/bayarea/article/City-First-In-Nation-To-Ban-Predictive-Policing-15872642.php. Accessed 25 May 2021.

  6. Geolitica.com. www.geolitica.com (accessed 25 May 2021)

  7. Los Angeles Police Department.: The Los Angeles Police Department Strategic Plan 2019–2021 (2019). http://lapd-assets.lapdonline.org/assets/pdf/Strategic%20Plan%202019-2021.pdf. Accessed 25 May 2021

  8. Ferguson, A.G.: The Rise of Big Data Policing: Surveillance, Race, and the Future of Law Enforcement. NYU Press, New York (2017)

    Book  Google Scholar 

  9. Dou, E.: China built the world’s largest facial recognition system. Now, it’s getting camera-shy. Wash. Post. https://www.washingtonpost.com/world/facial-recognition-china-tech-data/2021/07/30/404c2e96-f049-11eb-81b2-9b7061a582d8_story.html (accessed 9 Sep 2021)

  10. Woollacott, E.: UK Government Accused Of Sneaking Through New Live Facial Recognition Rules. Forbes. https://www.forbes.com/sites/emmawoollacott/2021/08/23/uk-government-accused-of-sneaking-through-new-live-facial-recognition-rules/ (accessed 9 Sep 2021).

  11. Bostrom, N.: The vulnerable world hypothesis. Glob. Policy 10, 455–476 (2019). https://doi.org/10.1111/1758-5899.12718

    Article  Google Scholar 

  12. Benjamin, R.: Race after technology: abolitionist tools for the new Jim code. Polity (2019). https://doi.org/10.1093/sf/soz162

    Article  Google Scholar 

  13. Brayne, S.: Predict and Surveil: Data, Discretion, and the Future of Policing. Oxford University Press, New York (2020)

    Book  Google Scholar 

  14. Lazer, D., Radford, J.: Data ex machina: introduction to big data. Annu Rev Sociol 43, 19–39 (2017)

    Article  Google Scholar 

  15. Krafft, T., Zweig, K.A., Konig, P.D.: How to regulate algorithmic decision-making: a framework of regulatory requirements for different applications. Regul. Gov. (2020). https://doi.org/10.1111/rego.12369

    Article  Google Scholar 

  16. Barocas, S., Rosenblat, A., Boyd, D., et al.: Data and civil rights: technology primer. Data Soc. https://datasociety.net/library/data-civil-rights-technology-primer/ (2014). (Accessed 12 May 2021)

  17. Short, M.B., D’Orsogna, M.R., Brantingham, P.J., et al.: Measuring and modeling repeat and near-repeat burglary effects. J. Quant. Criminol. 25, 325–339 (2009)

    Article  Google Scholar 

  18. Miller, L.: LAPD will end controversial program that aimed to predict where crimes would occur. Los Angel. Times. https://www.latimes.com/california/story/2020-04-21/lapd-ends-predictive-policing-program (2020). (Accessed 15 Feb 2021)

  19. Marda, V., Naravan, S.: Data in New Delhi's predictive policing system. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp 317–324

  20. Cohen, L.E., Kluegel, J., Land, K.: Social inequality and predatory criminal victimization: an exposition and test of formal theory. Am. Sociol. Rev. 46, 505–524 (1981)

    Article  Google Scholar 

  21. Caplan, J., Kennedy, L., Miller, J.: Risk terrain modeling: brokering criminological theory and GIS methods for crime forecasting. Justice Q. 28, 360–381 (2011)

    Article  Google Scholar 

  22. Uchida, C.D., Swatt, M., Gamero, D., et al.: The Los Angeles Smart Policing Initiative: Reducing Gun-Related Violence through Operation LASER. U.S. Department of Justice, Bureau of Justice Assistance. (2012). http://newweb.jssinc.org/wp-content/uploads/2014/11/Spotlight-on-Operation-LASER.pdf. Accessed 25 May 2021

  23. McGrory, K., Bedi, N.: Targeted. Tampa Bay Times. https://projects.tampabay.com/projects/2020/investigations/police-pasco-sheriff-targeted/intelligence-led-policing/ (2020). (Accessed 15 Feb 2021)

  24. Review of the MPS Gangs Matrix. Mayor’s Office of Policing and Crime 2018. https://www.london.gov.uk/mopac-publications-0/review-mps-gangs-matrix (accessed 25 May 2021)

  25. Review of the MPS Gangs Violence Matrix - Update. Mayor’s Office of Policing and Crime. https://www.london.gov.uk/mopac-publications-0/review-mps-gangs-violence-matrix-update (2021) (Accessed 25 May 2021)

  26. Ferguson, A.G.: Illuminating black data policing. Ohio State J. Crim. Law 15, 503–525 (2018)

    Google Scholar 

  27. Fussey, P., Murray, D.: Independent Report on the London Metropolitan Police Service’s Trial of Live Facial Recognition Technology. University of Essex: The Human Rights, Big Data, and Technology Project (2019). https://repository.essex.ac.uk/24946/1/London-Met-Police-Trial-of-Facial-Recognition-Tech-Report-2.pdf. Accessed 25 May 2021.

  28. Kostka, G., Steinacker, L., Meckel, M.: Between security and convenience: Facial recognition technology in the eyes of citizens in China, Germany, the United Kingdom, and the United States. Public Underst. Sci. 30(6), 671–690 (2021). (forthcoming)

    Article  Google Scholar 

  29. Chen, S.: Is China’s corruption-busting AI system “Zero Trust” being turned off for being too efficient? South China Morning Post. https://www.scmp.com/news/china/science/article/2184857/chinas-corruption-busting-ai-system-zero-trust-being-turned-being (2019). (Accessed 25 May 2021)

  30. Brey, P.: Ethical aspects of facial recognition systems in public places. J. Inf. Commun. Ethics Soc. 2, 97–109 (2004). https://doi.org/10.1108/14779960480000246

    Article  Google Scholar 

  31. Piquero, A., Brame, B.: Assessing the race-crime and ethnicity-crime relationship in a sample of serious adolescent delinquents. Crime Delinquency 54, 390–422 (2008)

    Article  Google Scholar 

  32. Harcourt, B.E.: Against Prediction: Profiling, Policing, and Punishing in an Actuarial Age. University of Chicago Press, Chicago (2006)

    Book  Google Scholar 

  33. Ensign, D., Friedler, S.A., Neville, S., et al.: Runaway Feedback Loops in Predictive Policing. In: Conference on Fairness, Accountability and Transparency. PMLR, pp. 160–71. http://proceedings.mlr.press/v81/ensign18a.html (2018). (Accessed 16 Apr 2021)

  34. Buolamwini, J., Gebru, T.: Gender shades: intersectional accuracy disparities in commercial gender classification. Proc. Mach. Learn. Res. 81, 1–15 (2018)

    Google Scholar 

  35. Boba, S.R.: Predictive policing: Where’s the evidence? In: Weisburd, D., Braga, A.A. (eds.) Police Innovation: Contrasting Perspectives. Cambridge University Press, Cambridge (2020)

    Google Scholar 

  36. Brantingham, J., Valasik, M.: Does predictive policing lead to biased arrests? Results from a randomized controlled trial. Stat. Public Policy 5, 1–6 (2018)

    Article  Google Scholar 

  37. Ekins EE. Policing in America: Understanding Public Attitudes Toward the Police. Results from a National Survey. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2919449 (2016). (Accessed 15 Feb 2021)

  38. Bobo, L.D., Thompson, V.: Unfair by design: the war on drugs, race, and the legitimacy of the criminal justice system. Soc. Res. 73, 445–472 (2006)

    Article  Google Scholar 

  39. Brooks, R.R.W.: Fear and fairness in the city: criminal enforcement and perceptions of fairness in minority communities. Calif. Law Rev. 73, 1228–1229 (2000)

    Google Scholar 

  40. Hagan, J., Albonetti, C.: Race, class, and the perception of criminal injustice in America. Am. J. Sociol. 88, 329–355 (1982)

    Article  Google Scholar 

  41. Saad, L.: Black Americans want police to retain local presence. Gallup. https://news.gallup.com/poll/316571/black-americans-police-retain-local-presence.aspx (2020). (Accessed 15 Feb 2021)

  42. Supervisory Powers of the Information Commissioner, Advisory Notice. https://www.met.police.uk/SysSiteAssets/media/downloads/force-content/met/about-us/gangs-violence-matrix/ico-enforcement-notice.pdf (Accessed 24 May 2021)

  43. Densley, J.A., Pyrooz, D.C.: The Matrix in context: taking stock of police gang databases in London and beyond. Youth Justice 20, 11–30 (2020)

    Article  Google Scholar 

  44. Brayne, S.: Surveillance and system avoidance: criminal justice contact and institutional attachment. Am. Sociol. Rev. 79, 367–391 (2014)

    Article  Google Scholar 

  45. Ferguson, A.G.: Big data and predictive reasonable suspicion. Univ. Pa. Law Rev. 163, 327–410 (2015)

    Google Scholar 

  46. Barocas, S., Selbst, A.D.: Big data’s disparate impact. Calif. Law Rev. 104, 671–732 (2016)

    Google Scholar 

  47. Corbett-Davies, S., Goel, S.: The Measure and Mismeasure of Fairness: A Critical Review of Fair Machine Learning. https://arxiv.org/abs/1808.00023 (2018). (Accessed 15 Feb 2021)

  48. Krafft, P.M., Young, M., Katell, M., et al.: An action-oriented AI policy toolkit for technology audits by community advocates and activists. In: Proceeding 2021 ACM Conference of Fairness Account of Transpiration. pp. 772–81 (2021)

  49. Los Angeles Police Commission.: Review of Selected Los Angeles Police Department Data-Driven Policing Strategies. Office of the Inspector General. (2019). https://www.oig.lacity.org/_files/ugd/b2dd23_21f6fe20f1b84c179abf440d4c049219.pdf. Accessed 25 May 2021.

  50. Selbst, A.: Disparate Impact in Big Data Policing. Ga Law Rev 52, 109–195 (2017)

    Google Scholar 

  51. Adams, M., Rameau, M.: Black community control over police. Wis. Law Rev. 515, 515–539 (2016)

  52. Vitale, A.S.: The End of Policing. Verso Books, New York (2017)

    Google Scholar 

  53. Muhammad, K.G.: The Condemnation of Blackness: Race, Crime, and the Making of Modern Urban America. Harvard University Press, Cambridge (2019)

    Book  Google Scholar 

  54. Capers, I.B.: Race, policing, and technology. NC Law Rev. 95 1241–1292 (2017)

  55. Ryan-Mosley T, Strong J. The activist dismantling racist police algorithms. MIT Technol. Rev. https://www.technologyreview.com/2020/06/05/1002709/the-activist-dismantling-racist-police-algorithms/ (2020). (Accessed 25 Feb 2021)

  56. Noble, S.U.: Algorithms of Oppression: How Search Engines Reinforce Racism. NYU Press, New York (2018)

    Book  Google Scholar 

  57. Dumke M, Main F. A look inside the watch list Chicago police fought to keep secret. Chic. Sun-Times. https://chicago.suntimes.com/2017/5/18/18386116/a-look-inside-the-watch-list-chicago-police-fought-to-keep-secret (2017). (Accessed 25 Feb 2021)

  58. Vestby, A., Vestby, J.: Machine learning and the police: asking the right questions. Polic.: J. Policy Pract. (2019). https://doi.org/10.1093/police/paz035

    Article  Google Scholar 

  59. Skeem, J., Lowenkamp, C.: Using algorithms to address trade-offs inherent in predicting recidivism. Behav. Sci. Law 38, 259–278 (2020)

    Article  Google Scholar 

  60. Edwards, L., Veale, M.: Slave to the algorithm? Why a “right to an explanation” is probably not the remedy you are looking for. Duke Law Technol. Rev. 16, 18–84 (2017)

    Google Scholar 

  61. Wachter, S., Mittelstadt, B., Floridi, L.: Transparent, explainable, and accountable AI for robotics. Sci. Robot. (2017). https://doi.org/10.1126/scirobotics.aan6080

    Article  Google Scholar 

  62. Kleinberg, J., Ludwig, J., Mullainathan, S., et al.: Discrimination in the age of algorithms. J. Leg. Anal. 10, 113–174 (2018)

    Article  Google Scholar 

  63. Zerilli, J., Knott, A., Maclaurin, J., et al.: Transparency in algorithmic and human decision-making: Is there a double standard? Philos. Technol. 32, 661–683 (2019). https://doi.org/10.1007/s13347-018-0330-6

    Article  Google Scholar 

  64. Beck, A., Blumstein, A.: Racial disproportionality in U.S. State prisons: accounting for the effects of racial and ethnic differences in criminal involvement, arrests, sentencing, and time served. J. Quant. Criminol. 34, 853–883 (2018)

    Article  Google Scholar 

  65. Blumstein, A.: Bringing down the U.S. prison population. Prison J. (2011). https://doi.org/10.1177/0032885511415218

    Article  Google Scholar 

  66. Goodman, B., Flaxman, S.: European Union regulations on algorithmic decision-making and a “right to explanation.” AI Mag. 38, 50–57 (2017)

    Google Scholar 

  67. Ferguson, A.G.: Policing predictive policing. Wash Univ. Law Rev. 94, 1109–1189 (2017)

    Google Scholar 

  68. Bakke, E.: Predictive policing: the argument for public transparency. Ann. Surv. Am. Law 74, 131–172 (2018)

    Google Scholar 

  69. Burrell, J.: How the machine “thinks”: understanding opacity in machine learning algorithms. Big Data Soc. (2016). https://doi.org/10.1177/2053951715622512

    Article  Google Scholar 

  70. State v. Loomis, 881 N.W.2d 749, 767 Wis. (2016)

  71. Tampa Bay Times.: Read the Pasco Sheriff’s Office response to our investigation. Tampa Bay Times. https://projects.tampabay.com/projects/2020/investigations/police-pasco-sheriff-targeted/sheriffs-response/ (2020). (Accessed 15 Feb 2021)

  72. Crump, C.: Surveillance policymaking by procurement. Wash Law Rev. (2016). https://doi.org/10.2139/ssrn.2737006

    Article  Google Scholar 

  73. Joh, E.: The undue influence of surveillance technology companies on policing. NU Law Rev. (2017). https://doi.org/10.2139/ssrn.2924620

    Article  Google Scholar 

  74. Fasciale, P.J.A.D.: State of New Jersey v. Corey Pickett. https://www.eff.org/files/2021/02/03/decision_appdiv_232021.pdf (2021)

  75. Wexler, R.: Life, liberty, and trade secrets: intellectual property in the criminal justice system. Stanford Law Rev. 70, 1343–1430 (2018)

    Google Scholar 

  76. Reisman, D., Schultz, J., Crawford, K., et al.: Algorithmic Impact Assessments: A Practical Framework for Public Agency Accountability. AI Now Institute https://ainowinstitute.org/aiareport2018.pdf (accessed 24 May 2021)

  77. Booker, C.: Booker, Wyden, Clarke Introduce Bill Requiring Companies To Target Bias In Corporate Algorithms. https://www.booker.senate.gov/news/press/booker-wyden-clarke-introduce-bill-requiring-companies-to-target-bias-in-corporate-algorithms (2019). (Accessed 15 Feb 2021)

  78. Wachter, S., Mittelstadt, B., Russell, C.: Counterfactual explanations without opening the black box: automated decisions and the GDPR. Harv. J. Law Technol. Harv. JOLT 31, 841 (2017)

    Google Scholar 

  79. Raji, I.D., Buolamwini, J.: Actionable Auditing: Investigating the Impact of Publicly Naming Biased Performance Results of Commercial AI Products. In: AIES 19 Proceeding 2019 AAAIACM Conference AI Ethics Society. pp. 429–35 (2019)

  80. Policing Project. www.policingproject.org (2020). (Accessed 26 Feb 2021)

  81. Engler AC. Independent auditors are struggling to hold AI companies accountable. Fast Co. https://www.fastcompany.com/90597594/ai-algorithm-auditing-hirevue (2021). (Accessed 15 Feb 2021)

  82. ACLU. Community Control Police Surveill. TakeCTRL. https://communityresourcehub.org/resources/community-control-over-police-surveillance-takectrl/ (2018). (Accessed 26 Feb 2021)

  83. Ferguson, A.G.: It’s time for D.C. to regulate police surveillance technology. Wash. Post. https://www.washingtonpost.com/opinions/local-opinions/its-time-for-dc-to-regulate-police-surveillance-technology/2020/06/25/9e94feb6-b57a-11ea-aca5-ebb63d27e1ff_story.html (2020). (Accessed 26 Feb 2021)

Download references

Acknowledgements

This material is based upon work supported by the National Science Foundation under Grant No. 1917707. Support for Dr. Davis’s work on the project was provided by the UF College of Journalism and Communications Consortium for Trust in Media and Technology. We also want to thank Ryan Jenkins for comments on earlier drafts of this paper.

Funding

This work is supported by the National Science Foundation under Grant No. 1917707, and the UF College of Journalism and Communications Consortium for Trust in Media and Technology.

Author information

Authors and Affiliations

  1. Department of Philosophy, University of Florida, Gainesville, FL, USA

    Jeremy Davis, Duncan Purves & Schuyler Sturm

  2. Computer and Information Science and Engineering Department, University of Florida, Gainesville, FL, USA

    Juan Gilbert

Authors
  1. Jeremy Davis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Duncan Purves
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Juan Gilbert
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Schuyler Sturm
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jeremy Davis.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Ethics approval

Not applicable.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Davis, J., Purves, D., Gilbert, J. et al. Five ethical challenges facing data-driven policing. AI Ethics 2, 185–198 (2022). https://doi.org/10.1007/s43681-021-00105-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s43681-021-00105-9

Keywords

Search

Navigation