Abstract
Hot-spot maps regularly aid many policing resource allocation decisions in today’s data-driven age. However, it is unclear what forecasting algorithm(s) should be used to create these maps. To address this gap, we must be able to assess how “good” a generated hot-spot map is. Currently, four main metrics are used for evaluation: the prediction accuracy index (PAI), the recapture rate index (RRI), the prediction efficiency index (PEI), and the prediction efficiency index* (PEI*). This article discusses PAI, RRI, and PEI’s strengths and weaknesses, articulates and justifies PEI*, and demonstrates the differences in calculations and interpretations of each metric. We argue that PEI* measures the efficiency of a crime forecasting algorithm while being operationally realistic and should be used in conjunction with other appropriate measures.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bowers, Kate J., Shane D. Johnson, and Ken Pease. 2004. Prospective Hot-Spotting: The Future of Crime Mapping? The British Journal of Criminology 44 (5): 641–658. https://doi.org/10.1093/bjc/azh036.
Braga, Anthony, Andrew Papachristos, and David Hureau. 2012. Hot Spots Policing Effects on Crime. Campbell Systematic Reviews 8 (1): 1–96. https://doi.org/10.4073/csr.2012.8.
Braga, Anthony, Brandon Turchan, Andrew V. Papachristos, and David M. Hureau. 2019. Hot Spots Policing of Small Geographic Areas Effects on Crime. Campbell Systematic Reviews 15 (3): e1046. https://doi.org/10.1002/cl2.1046.
Butt, Umair Muneer, Sukumar Letchmunan, Fadratul Hafinaz Hassan, Mubashir Ali, Anees Baqir, Tieng Wei Koh, and Hafiz Husnain Raza. Sherazi. 2021. Spatio-Temporal Crime Predictions by Leveraging Artificial Intelligence for Citizens Security in Smart Cities. IEEE Access 9: 47516–47529. https://doi.org/10.1109/ACCESS.2021.3068306.
Chainey, Spencer, Lisa Tompson, and Sebastian Uhlig. 2008. The Utility of Hotspot Mapping for Predicting Spatial Patterns of Crime. Security Journal 21 (1): 4–28. https://doi.org/10.1057/palgrave.sj.8350066.
Drawve, Grant. 2016. A Metric Comparison of Predictive Hot Spot Techniques and RTM. Justice Quarterly 33 (3): 369–397. https://doi.org/10.1080/07418825.2014.904393.
Drawve, Grant, and Alese Wooditch. 2019. 2019_A Research Note on the Methodological and Theoretical Considerations for Assessing Crime Corecasting Accuracy with the Predictive Accuracy Index_Drawve Wooditch.PDF. Journal of Criminal Justice 64: 101625.
Egami, Naoki, and Erin Hartman. 2022. Elements of External Validity: Framework, Design, and Analysis. Design and Analysis. https://doi.org/10.2139/ssrn.3775158.
Fan, Shuzhan. 2014. The Spatial-Temporal Prediction of Various Crime Types in Houston, TX Based on Hot-Spot Techniques. Baton Rouge: Louisiana State University.
Galvis, Juan, Freddy Hernández-Romero, and Francisco Gómez. 2022. A Note on Averaging Prediction Accuracy, Green’s Functions and Other Kernels PREPRINT (Version 1) (February). https://doi.org/10.21203/rs.3.rs-1348136/v1
Gong, Junfang, Shengwen Li, and Bo. Wan. 2019. A Regional Approach to Assessing and Visualizing Spatiotemporal Clustering of Crime Events. Papers in Applied Geography 5 (1–2): 26–44. https://doi.org/10.1080/23754931.2019.1611625.
Groff, Elizabeth R., and Nancy G. La Vigne. 2002. Forecasting the Future of Predictive Crime Mapping. Crime Prevention Studies 13: 29–58.
Hart, Timothy C. 2020. Hot Spots of Crime: Methods and Predictive Analytics. In Geographies of Behavioural Health, Crime, and Disorder: The Intersection of Social Problems and Place, GeoJournal Library, ed. Kim M. Lersch, Jayajit Chakraborty, and GeoJournal Library, 87–103. Cham: Springer. https://doi.org/10.1007/978-3-030-33467-3_5.
Hunt, Joel M. 2016. “Do Crime Hot Spots Move? Exploring the Effects of the Modifiable Areal Unit Problem and Modifiable Temporal Unit Problem on Crime Hot Spot Stability.” Doctoral Dissertation, United States -- District of Columbia: American University. https://search.proquest.com/docview/1831606816/abstract/B1156D800B9349D8PQ/1.
Jefferis, Eric. 1999. A Multi-Method Exploration of Crime Hot Spots: A Summary of Findings. Washington, DC: US Department of Justice National Institute of Justice Crime Mapping Research Center
Joshi, Chaitanya, Sophie Curtis-Ham, Clayton D’Ath, and Deane Searle. 2021. Considerations for Developing Predictive Spatial Models of Crime and New Methods for Measuring Their Accuracy. ISPRS International Journal of Geo-Information 10 (9): 597. https://doi.org/10.3390/ijgi10090597.
Kounadi, Ourania, Alina Ristea, Adelson Araujo, and Michael Leitner. 2020. A Systematic Review on Spatial Crime Forecasting. Crime Science 9 (1): 7. https://doi.org/10.1186/s40163-020-00116-7.
Kruskal, J.B. 1956. On the Shortest Spanning Subtree of a Graph and the Traveling Salesman Problem. Proceedings of the American Mathematical Society 7 (1): 48–50.
Lee, YongJei, and O. SooHyun. 2020. Flag and Boost Theories for Hot Spot Forecasting: An Application of NIJ’s Real-Time Crime Forecasting Algorithm Using Colorado Springs Crime Data. International Journal of Police Science & Management 22 (1): 4–15. https://doi.org/10.1177/1461355719864367.
Lee, YongJei, O. SooHyun, and John E. Eck. 2020. A Theory-Driven Algorithm for Real-Time Crime Hot Spot Forecasting. Police Quarterly 23 (2): 174–201. https://doi.org/10.1177/1098611119887809.
Levine, N. 2008. “The ‘Hottest’ Part of a Hotspot: Comments on ‘The Utility of Hotspot Mapping for Predicting Spatial Patterns of Crime.’” Security Journal 21, no. 4 (October 1, 2008): 295–302. https://doi.org/10.1057/sj.2008.5
National Insitute of Justice (NIJ). 2017. Real-Time Crime Forecasting Challenge Posting. Washington, DC: National Institute of Justice.
Rashidi, Parinaz, Tiejun Wang, Andrew Skidmore, Anton Vrieling, Roshanak Darvishzadeh, Bert Toxopeus, Shadrack Ngene, and Patrick Omondi. 2015. Spatial and Spatiotemporal Clustering Methods for Detecting Elephant Poaching Hotspots. Ecological Modelling 297 (February): 180–186. https://doi.org/10.1016/j.ecolmodel.2014.11.017.
Van Patten, Isaac T., Jennifer McKledin-Coner, and Deana Cox. 2009. A Microspatial Analysis of Robbery: Prospective Hot Spotting in a Small City. Crime Mapping. http://ivanpatt.asp.radford.edu/Research/Microspatial%20Analysis%20of%20Robbery.pdf. Accessed: 30 Dec 2022
Warren L. G. Koontz. 2017. Analysis and Prediction of Call For Service Data. 251177. 2016-NIJ-Challenge-0021.
White, Veronica M., and Joel M. Hunt. 2022. Measuring How Relatively ‘Good’ a Hot-Spot Map Is: A Summary of Current Metrics. In Proceedings of the IISE Annual Conference & Expo 2022, edited by K. Ellis, J. Knapp, and W. Ferrell. Seattle, WA. https://nij.ojp.gov/library/publications/measuring-how-relatively-good-hot-spot-map-summary-current-metrics. Accessed: 1 Jan 2022
Disclaimer
The opinions, findings, conclusions, and recommendations expressed in this publication are those of the authors and do not necessarily reflect the official position or policies of the U.S. Department of Justice.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Disclaimer The opinions, findings, conclusions, and recommendations expressed in this publication are those of the authors and do not necessarily reflect the official position or policies of the U.S. Department of Justice.
Rights and permissions
About this article
Cite this article
White, V.M., Hunt, J. & Green, B. A discussion of current crime forecasting indices and an improvement to the prediction efficiency index for applications. Secur J 37, 47–64 (2024). https://doi.org/10.1057/s41284-023-00367-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1057/s41284-023-00367-4